Skip to Content
SourcesFile Formats

Supported File Formats

Connect a system and Classifyre reads what’s actually in it — not just the plain text. The scanned contract, the spreadsheet attached to a ticket, the screenshot pasted into a chat thread, the images packed inside a dataset: each becomes content your detectors can search.

This page answers the two questions that come up first: what can Classifyre read, and which sources do those files come from?

The format is worked out from the file’s contents, not just its name — so a mislabelled export, a file with no extension, or a document a system reports as “binary” is still read as what it really is. That matters in practice: cloud storage tends to label almost everything as generic binary data.


Which file formats does Classifyre support?

Content typeFile typesWhat Classifyre gets out of it
DocumentsPDF, DOCX, PPTX, RTF, ODT, ODP, and legacy DOC / XLS / PPTThe full text, including tables — with OCR for pages that are scans
SpreadsheetsXLSX, ODSEvery sheet, row by row — the header is found even under a title row
Tables & datasetsCSV, TSV, JSON Lines (JSONL / NDJSON), Parquet, Arrow, Feather, IPC, Avro, ORC, VortexEvery row, with columns labelled so a finding points at a real record
Database filesSQLite (.sqlite, .db), DuckDB, GeoPackageEvery table, row by row — a database someone left in a bucket is read like one
GeospatialGeoJSON, FlatGeobuf, Shapefile attribute tables (.dbf); KML, GPX and GML as XMLEvery feature’s attributes, with its coordinates as text
Text & dataTXT, Markdown, JSON, XML, HTML, log filesThe content as written, with encoding detected automatically
Code & configPython, JavaScript / TypeScript, Java, Go, Rust, C / C++, C#, Ruby, PHP, Swift, Kotlin, shell, SQL, Terraform, YAML, TOML, INI, .env, Dockerfile, MakefileEvery line, exactly as written — including the comments
EmailEML, MSGSender, recipients, subject, body, and attachment names
ImagesPNG, JPG, TIFF, BMP, WEBP, GIF, HEIC / HEIFText inside the image, via OCR
AudioMP3, WAV, M4A, AAC, OGG, OPUS, FLACA transcript of what was said
VideoMP4, MOV, MKV, WEBM, AVI, WMV, FLVA transcript and the text visible on screen
ArchivesZIP, TAR, GZ, TGZ, 7Z, RAREach file inside, unpacked and scanned on its own

Images, audio, and video go through OCR and transcription automatically — no switch needed. They cost more time per item than reading text, so exclude paths or extensions you don’t need searched in the source’s configuration.


Tables, whatever they were saved as

Every format with rows reaches your detectors the same way: one record at a time, each value labelled with its column. A phone number found in row 4,812 of an Avro file reads exactly like one found in a CSV or a database table, so column-aware detectors and the findings they produce look alike across formats.

A few things are worked out for you rather than assumed:

  • Delimiter, quoting and header. A TSV, a ;-separated export from a European locale, a pipe-delimited dump — each is split on its own delimiter. Lines above the header, such as Report generated on 2026-01-01, are recognised as a preamble rather than taken for the header.
  • Values stay as written. Cells are read as text, so 06641234567 stays a phone number and 00123 stays an ID, instead of losing their leading zeros.
  • Ragged rows keep every value. A short row is padded; a long one keeps its extra values in an extra_values column instead of dropping them.
  • Any text encoding. UTF-8, UTF-16, Latin-1, Windows code pages — the encoding is detected and the file read in it.
  • Records with different fields. In a JSON Lines file every field is read, including one that only appears in the very last record. Nested objects are shown as JSON.
  • Spreadsheet headers under title rows. The header row is found even when a title or a blank line sits above it, and every sheet is read. Columns are labelled Sheet.column.
  • Every table in a database file. SQLite, DuckDB and GeoPackage files are read table by table, with columns labelled table.column. Only tables are read — never views — and engine bookkeeping such as SQLite’s catalog or GeoPackage’s index tables is left out.
  • Coordinates as text. In geospatial files the geometry is written out as text (POINT (16.37 48.21)), so a location is something a detector can see.

Lakehouse tables in a bucket

A Delta Lake or Apache Iceberg table is a folder of files, but no single file is the table. Deleted rows can still sit in files the table no longer uses, older snapshots keep their own files, and a renamed column keeps its old name in every file written before the rename.

So when a cloud-storage or folder scan meets a table, it reads it as one table asset: current rows only, deletions applied, renamed columns as one column. Its data files are not scanned again one by one. A table is recognised by its own markers:

FormatRecognised by
Delta Lakea _delta_log/ folder
Apache Icebergmetadata/*.metadata.json
Apache Hudia .hoodie/ folder
Apache Paimonsnapshot/ and schema/ folders
Lancea *.lance folder with _versions/

A table that carries both Delta and Iceberg metadata (Delta UniForm) is read as Delta. Which formats can be read on which storage, and what to use when tables sit behind a catalog or an engine, is on Lakehouse Tables.


Source code and configuration

Code files are read as the text they are, so a key committed to config/settings.py, a password in values.yaml, or a live token in .env is found the same way one in a spreadsheet is. Roughly eighty extensions are recognised, plus the files that carry no extension at all — Dockerfile, Makefile, Jenkinsfile, Gemfile, .env.production.

This matters more than it sounds. A Python module whose first line is from typing import Any, Dict, List has two commas on line one, which is exactly what a spreadsheet’s header row looks like — and a minified JavaScript bundle starts with {, which is exactly what a JSON file looks like. Read by content alone, both get sent to a table reader that produces columns of nonsense and finds nothing. Classifyre uses the file name to settle it, so the file is read line by line as source.

The most common places a real credential turns up are the ones you would expect: .env and its per-environment variants, Terraform variable files, CI pipeline definitions, Kubernetes manifests, and connection strings in application config. All of them are read.


Files hidden inside other files

A lot of sensitive data doesn’t sit in a file — it sits in a file inside a file. Classifyre unpacks those and treats each one as a separate asset, with its own detectors, findings, and link back to the file it came from.

Where it hidesExample
Inside an archiveA ZIP in cloud storage holding 40 customer spreadsheets
Inside a dataset’s rowsAn ML dataset whose image or document column holds whole files
Inside a database fileA SQLite file whose BLOB column stores scanned contracts
Inside an Office documentScreenshots embedded in a slide deck or a spreadsheet

In the row text, an embedded file shows up as a compact placeholder such as <image: 41 KB> — the bytes themselves are scanned as their own asset, so a passport photo in row 4,000,001 gets flagged as an image finding rather than disappearing into a wall of unreadable characters.

Two things worth knowing:

  • Unpacking happens for file-based sources — cloud storage, local folders, direct uploads, Dropbox, and Hugging Face. A ZIP attached to a Jira ticket is catalogued and tracked, but its contents aren’t split out.
  • Unpacking follows your sampling settings — the files inside the rows a run reads become assets on that run. Fan-out is capped per file, so one dataset can’t turn into a million assets overnight.

Large files and very big tables

Big files aren’t truncated, and they aren’t loaded into memory whole. Classifyre streams them, so a multi-gigabyte export is read the same way a small one is.

For anything with rows — CSV, JSON Lines, Parquet, Arrow, Avro, ORC, spreadsheets, database files, database tables — your sampling strategy also governs how much of each file a single run reads:

StrategyWhat one run reads of a single table
AutomaticThe next slice of rows, resuming where it left off
LatestThe first rows of the file
RandomA different sample each run
AllEverything

With Automatic, a 40-million-row Parquet file is covered slice by slice across scheduled runs instead of blocking one enormous scan, and progress is tied to the file’s contents — if the file changes, the sweep restarts rather than resuming into rows that have moved.

For Parquet and Arrow files held in cloud storage, only the part of the file a run actually needs is fetched. A multi-gigabyte dataset shard can be scanned without ever being downloaded in full — which is what keeps egress costs and scan times predictable on large data lakes.


Which sources do files come from?

Everything above applies to these sources — this is where documents, images, media, and archives enter Classifyre.

SourceWhat it brings
S3-compatible storage, Azure Blob, Google Cloud StorageEvery object in a bucket or container
Local folderA directory tree on the machine running the scan
Git repositoryEvery file on one branch — code, config, docs, and committed data
SandboxFiles uploaded straight into Classifyre
DropboxFiles and shared folders
Hugging FaceDataset and model repository files, including large shards
Confluence, Jira, Jira Service ManagementPages, issues, requests, comments — and their attachments
NotionPages, data sources, comments, and uploaded files
SlackMessages and shared files
EmailMessages and every attachment
Google WorkspaceDrive files, Docs, Sheets, and Slides
Microsoft 365SharePoint, OneDrive, and Teams files
RedditPosts, comments, and attached media

Sources that bring records rather than files

These systems produce rows and documents instead of files. They’re read directly — no file parsing involved — and the same sampling controls apply.

CategorySources
DatabasesPostgreSQL, MySQL, SQL Server, Oracle, MongoDB
GraphNeo4j
SearchElasticsearch, OpenSearch, Meilisearch
Warehouse & lakehouseSnowflake, Databricks, Dremio, Hive, Delta Lake, Iceberg
StreamingKafka
Analytics & BIPower BI, Tableau
Web & socialWordPress, YouTube

The full, always-current list — with every configuration field per system — is in the source catalog.


Controlling what gets picked up

Reading everything is rarely what you want. Most file-based sources let you restrict the scan before anything is downloaded:

  • By file type — include or exclude extensions, so a bucket of build artefacts doesn’t consume a scan meant for documents.
  • By path — pattern filters such as data/**/*.parquet on object storage.
  • By scope — specific folders, projects, channels, labels, or mailboxes, depending on the system.

Filtering by name is far cheaper than downloading and discarding, so it’s the first lever to reach for on a large source. See Configuration & Fields for what each source supports.


Common questions

Does Classifyre read scanned PDFs? Yes. PDFs with a text layer are read directly; pages that are images get OCR automatically, so scanned contracts and faxes are searchable. No switch needed.

Can it read files inside a ZIP? Yes — for file-based sources, each file in the archive becomes its own asset with its own findings. Archives nested inside archives are catalogued but not opened recursively.

Does it handle Parquet and Arrow datasets? Yes, row by row, including columns that hold whole files such as images or documents. Large shards in cloud storage are read in place rather than downloaded.

Does it read Avro, ORC or JSON Lines? Yes, all three, row by row, with the same column labels as any other table. Columns holding whole files, such as an Avro field with image bytes, become assets of their own.

What about a SQLite or DuckDB file someone uploaded? It’s opened read-only and every table in it is scanned. Views and macros inside the file are never run, so a crafted database can’t make the scan read anything else.

Can I query these files with SQL? The table files DuckDB can read straight from a bucket — Parquet, CSV, TSV, JSON, JSON Lines, Avro and the geospatial formats — can, joined with your databases. Spreadsheets, SQLite and DuckDB files and ORC are scanned but not yet queryable; the full list is under SQL.

My bucket holds Delta or Iceberg tables. Why weren’t their Parquet files scanned one by one? Because a table’s files include rows the table has deleted. The scan reads each table as one asset instead, as it is now — see Lakehouse Tables.

Does it read source code? Yes — around eighty languages and configuration formats, plus extension-less files such as Dockerfile and Makefile. Each is read line by line as text, so a secret in a config file is found the same way one in a document is.

What about audio and video? Both are transcribed automatically, and video also has the text on screen read via OCR. No switch needed.

Will very large files break a scan? No. Files are streamed rather than loaded whole, and tables are read in bounded slices set by your sampling strategy.

What happens to a file type that isn’t supported? It’s still catalogued with its metadata and tracked across scans — it simply has no text for detectors to read.


Next steps

Last updated on