5000 Character Limit for Model Grader Payload

[Decadz]

Technical Feedback

Hi,

It looks like the Model Grader has a 5000-character limit for the entire payload. I suspect this might be unintentional, since @leestott mentioned that I could split the grader into multiple messages under 5000 characters each and it should still work. However, when I tried splitting the long prompt into several smaller messages (each validated programmatically to be under 5000 characters), the system still returned the 5000-character limit error. When I truncated the full payload to exactly 5000 characters, the training session launched successfully.

EDIT: The 5000-character limit is not just for model graders, but also python graders.

Desired Outcome

Let the model grader use longer prompts, or let user split multiple messages into <5000k chuncks.

Current Workaround

No current work around.