Organizations accumulate documents over the years, often without a consistent structure and in a variety of formats. While modern AI systems (e.g.,\u00a0agentic assistants, RAGs) provide new, powerful ways to speed up human workflows, their output is often bound by the quality of the input data. <\/p>\n\n\n\n
After years of growing the archives, their owners not only lose any organized structure, but often don’t even know what kind of documents they contain. This is a poor starting point for an effective AI transformation: not knowing the data makes planning hard, and the messiness is likely to hurt performance.<\/p>\n\n\n\n
This project investigates a lightweight approach to discovering the kinds of data a given corpus contains and to finding structure in a messy document store. This could be useful on its own or as a first step toward automatically (or semi-automatically) organizing such repositories and enriching them with metadata.<\/p>\n\n\n\n
As a proof of concept, we keep the scope narrow and:<\/p>\n\n\n\n
\n
focus on how much structure can be recovered from documents in an unsupervised manner<\/li>\n\n\n\n
go beyond text and experiment with representations based on layout and visual form<\/li>\n\n\n\n
explore grouping documents in an interpretable way and making the inferred structure easily readable<\/li>\n<\/ul>\n\n\n\n
Background<\/h2>\n\n\n\n
Topic modelling<\/strong><\/h3>\n\n\n\n
Finding structure in a text corpus is not a new problem, and has historically been solved with methods based on word statistics.<\/p>\n\n\n\n
A classic example is LDA (Latent Dirichlet Allocation): each document is treated as a mixture of topics, and each topic as a distribution over words, inferred from which words tend to appear together. It remains a useful tool today, but it has limitations. It only sees text, so anything carried by layout or visual form is invisible to it. It also focuses on word or phrase counts, so it might miss more complex semantic signals that modern language models are built to retrieve.<\/p>\n\n\n\n
Clustering asks a different question. Instead of a mixture of topics per document, it groups similar documents based on a representation that is not limited to bag-of-words.<\/p>\n\n\n\n
Embedding based clustering<\/strong><\/h3>\n\n\n\n
Modern approaches cluster documents in an embedding space (usually a neural embedding, though TF-IDF can also work), following roughly:<\/p>\n\n\n\n
A concrete example of this approach applied to text is BERTopic, which uses sentence embeddings \u2192 UMAP \u2192 HDBSCAN, followed by the extraction of topic representations based on word frequency.<\/p>\n\n\n\n
We follow a similar general pipeline for the clustering part, but focus on the choice of embeddings.<\/p>\n\n\n\n
Beyond text<\/strong><\/h3>\n\n\n\n
Documents carry more information than just the text’s semantics: layout and visual style are valuable signals and may be exactly what one wants to group by. We therefore look at how the choice of embedding shapes the resulting clusters and how it influences which signals surface.<\/p>\n\n\n\n
Cluster interpretability<\/strong><\/h3>\n\n\n\n
Both LDA and BERTopic describe the resulting clusters (topics) using representative words or phrases. That has its practical uses, but it does not make the clusters easy to interpret or organize. One can read into a word list and guess the theme of a group, but it takes effort, and such representation does not give much to base a structure on. The examples below show what this looks like in practice.<\/p>\n\n\n\n
Both examples come from our synthetic corpus (described below under Data).<\/p>\n\n\n\n
![\"top](\"https:\/\/sii.pl\/blog\/wp-content\/uploads\/2026\/10\/1-EN.png\")