Troubleshooting

Common Issues with Calk AI Agents

LLMs (the AI models behind your agents) are powerful but also a bit… annoying. They don’t always behave exactly how you’d like out of the box. That’s why in Calk AI, you’ll sometimes need to set things up yourself—whether that’s picking the right model, guiding it with the right prompts, or organizing your knowledge base properly.

Below are some of the most common issues you might encounter when working with your agents in Calk AI, along with practical ways to fix them.

“It’s not finding the right source”

• Focus on Specific Knowledge: Limit the agent’s access to only the most relevant knowledge files or folders. For example, if you have a Process Agent, connect it only to “Process” documents.

• Guide with Prompts: Add context in your prompt, such as “look for slides titled X” or “refer to documents that mention Y.”

• Organize Knowledge: Keep your knowledge base clean. Create folders or categories for specific topics so the agent can navigate them more easily.

“It’s making things up” (Hallucinations)

• Pick the Right Model: For complex or technical tasks, use a more advanced model like GPT-4.1 or Claude 3.7 instead of lightweight ones.

• Start Fresh: Very long conversations can cause drift. If the agent starts hallucinating, open a new chat.

• Step-by-Step Instructions: Use chain-of-thought prompting (“first do this, then do that”) to guide reasoning.

• Be Explicit: Tell the agent what not to do (“do not speculate” / “avoid assumptions”).

• Allow “I don’t know”: Add a line in your prompt: “If you don’t have the answer, simply reply: I don’t know.”

• Control Creativity: In advanced settings, adjust the creativity from Super creative to close to Serious to reduce fabrications. (We recommend “0.25”)

“It’s not giving me consistent output formats”

• Prompt the Format: Be clear about how you want the answer structured, e.g., “present this in a 3-column table with A, B, and C.”

• Provide Examples: Show the agent what a correct output looks like and ask it to copy that structure.

👉 Pro tip: Don’t hesitate to test multiple models in Calk’s multi-model chat. Different models handle reasoning, formatting, or summarization differently—try a few until you find the one that best fits your workflow.