How to check a JSONL dataset before fine-tuning
A fine-tuning job reads JSONL one line at a time, and one malformed line is enough to fail the upload after you have already paid for the wait. Checking the file first takes seconds.
What actually breaks
- A record that is not valid JSON. Usually a stray newline inside a record, so one logical example is split across two lines and neither parses.
- A record that is not an object. A bare string or list on its own line.
- Duplicate keys in one record. JSON parsers keep the last one silently, so a record with two
contentkeys trains on a value you never intended. - Exact duplicate records. Repeated examples quietly reweight the dataset.
- Non-finite numbers.
NaNandInfinityare accepted by some writers and rejected by strict readers. - Chat-format mistakes. A missing
messageslist, an unknown role, empty content, no assistant message, or roles that do not alternate after the leading system message.
A quick manual check
Parsing every line locally catches the first two classes and nothing else.
python3 -c "import json,sys [json.loads(l) for l in open(sys.argv[1]) if l.strip()]" dataset.jsonl
It stops at the first failure, says nothing about duplicates or overwritten keys, and does not know what a conversational record is supposed to look like.
Checking the whole file at once
The JSONL Dataset Validator reads the dataset line by line and returns every issue it finds with the line number, a stable issue code, and a severity, rather than stopping at the first one. Choose the chat format to add the role and content rules, or leave it generic to check structure only.
It never echoes the content of a record back, so an issue list can be pasted into a ticket without leaking the dataset. One run handles up to 500 records and 48 KB of input, so split a large corpus and validate it in parts.
Reading the result
- error
- The line will fail or train on something other than what it says. Fix it.
- warning
- The line is valid but suspicious, such as an unusual role order.
- line
- The 1-based line in the file you uploaded, so the fix is a direct lookup.
Next
- Validate a dataset now.
- Single records failing to parse? Start with fixing a trailing comma in JSON.