https://www.simplifylegalintelligence.com/legal-robots daily 0.75 2025-05-02 https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/c7e0f485-1069-43f1-80ab-2877e8bce8b5/Scribble+Logo+%28black+background%29v3.png https://www.simplifylegalintelligence.com/legal-robots/foundation-data-llm-performance monthly 0.5 2025-05-02 https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/bf7fdfd6-5ce1-46d2-8fa1-0e00a975e745/SLI+-+Foundation+Data+Simplified+%284-15-25%29.png Legal Robots: AI Insights for Legal Professionals - Foundation Data In Action: Closing the “Last 10%” Gap in LLM Performance - Make it stand out Whatever it is, the way you tell your story online can make all the difference. https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/d67047b8-a8c4-4582-a594-23a5c00ee7b2/SLI+-FAQ+Workflow+%285-2-25%29.jpg Legal Robots: AI Insights for Legal Professionals - Foundation Data In Action: Closing the “Last 10%” Gap in LLM Performance https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/99b13115-8e8a-4e2b-8d84-d092b491c0fd/Albus+-+FAQ+Side+by+Side+%284-14-25%29%28annotated%29v2.png Legal Robots: AI Insights for Legal Professionals - Foundation Data In Action: Closing the “Last 10%” Gap in LLM Performance - Make it stand out An out-of-the-box LLM and a Foundation Data-trained LLM were each asked to create an FAQ outline for SLI’s website using nearly identical prompts (e.g., I didn’t say “Hey Albus” to the generic model). https://www.simplifylegalintelligence.com/legal-robots/llm-limitations-legal-ai monthly 0.5 2025-05-02 https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/4f8f0f03-19c4-49ff-8062-29e33cce00d0/AI+Style+Image+%28final%29%284-8-25%29+%28white+bg%29.png Legal Robots: AI Insights for Legal Professionals - Problems LLMs (Currently) Have: Navigating Limitations in Legal AI - Formulaic output: LLMs often produce text that is overly polished and prone to repetitive phrasing, sometimes derisively called “AI slop”. Provide a stylebook with voice, tone, and formatting guidelines to get better first drafts Human refinement as needed (and it will be needed) https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/7daf8893-4550-4ebc-adbc-c234dea28a28/Lost+in+the+Middle+Chart.png Legal Robots: AI Insights for Legal Professionals - Problems LLMs (Currently) Have: Navigating Limitations in Legal AI - When models are asked to retrieve specific information (the “needle in a haystack”) from long context, research finds a “Lost in the Middle” effect. This U-shaped performance curve shows that LLMs perform better when the “needle” is at the beginning or end of the input, but struggle when it’s in the middle. This happens because their attention mechanisms have both primacy and recency bias (position matters). Figure: Accuracy of language model in retrieving relevant documents by position of the target document. Source: Liu et al. (2024). "Lost in the Middle: How Language Models Use Long Contexts." Transactions of the Association for Computational Linguistics, 12, 157–173. Available at https://doi.org/10.1162/tacl_a_00638. https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/bba70fe0-4120-49ac-b216-664306fa925b/Long+Context+Chart.png Legal Robots: AI Insights for Legal Professionals - Problems LLMs (Currently) Have: Navigating Limitations in Legal AI - When models are asked to apply complex reasoning or infer associations over long context, performance tends to degrade more uniformly as context length increases (total context length matters). Figure: Model performance across different context lengths in the NoLiMa benchmark. Source: Modarressi et al. (2025). "NoLiMa: Long-Context Evaluation Beyond Literal Matching." arXiv:2502.05167 [cs.CL]. Available at https://arxiv.org/abs/2502.05167. https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/e6007c99-7ddd-4429-b594-64ebcf354799/Task+Switching+%284-9-25%29.png Legal Robots: AI Insights for Legal Professionals - Problems LLMs (Currently) Have: Navigating Limitations in Legal AI - Research has documented this “task interference” effect. The model's internal state carries over signals from one task that bias or confuse its approach to subsequent tasks. Figure: illustrative example of a task-switch which results in an incorrect response. Source: Gupta et al. (2024). "LLM Task Interference: An Initial Study on the Impact of Task-Switch in Conversational History." arXiv:2402.18216v2 [cs.CL]. Available at https://arxiv.org/html/2402.18216v2. https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/77dac08e-5489-4b8b-b352-783cf6b3240e/Needle+in+a+Haystack+Image+%284-8-25%29%28white+background%29.png Legal Robots: AI Insights for Legal Professionals - Problems LLMs (Currently) Have: Navigating Limitations in Legal AI - Needle in a haystack: LLMs can miss specific information in long contexts or prompts. Address this by: dividing any task that would require more than 20 pages of input into smaller steps distilling key information into structured context directing the model's attention to specific sections implementing orchestrated workflows https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/6107632b-3159-4055-9ccd-2a5d81df578b/Multi-tasking+Image+%284-8-25%29%28white+bg%29.png Legal Robots: AI Insights for Legal Professionals - Problems LLMs (Currently) Have: Navigating Limitations in Legal AI - Multi-task processing: LLMs can do many kinds of things, but they perform much better when doing one kind of thing at a time. Break complex tasks into discrete steps with specialized prompts and targeted context for each stage. Work toward implementing orchestrated workflows. https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/90019bc7-1662-44e7-a6ad-32a4c804210e/Thumbnail+Image+%284-8-25%29.png Legal Robots: AI Insights for Legal Professionals - Problems LLMs (Currently) Have: Navigating Limitations in Legal AI - Adult supervision required Legal professionals need a clear-eyed view of where LLMs struggle so they can navigate the pitfalls that come from misplaced confidence. Below I’ll go over some limitations in the current generation of models that I consider most impactful in legal contexts, and strategies I’ve found helpful in managing them day-to-day. https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/a1d23f1d-da26-43ec-bd44-6bfc8ae3342b/Sycophancy+Image+2+%284-8-25%29%28white+background%29.png Legal Robots: AI Insights for Legal Professionals - Problems LLMs (Currently) Have: Navigating Limitations in Legal AI - Sycophancy: LLMs often agree with users by default, even when incorrect. Mitigate this through: neutral prompt framing explicit instructions to challenge assumptions structured context https://www.simplifylegalintelligence.com/legal-robots/category/AI+Simplified monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/category/Legal+Technology monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/category/Foundation+Data monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/category/Prompts monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/category/Agents monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/category/AI+Implementation monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/category/Models monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/category/Data monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/category/Foundation+Services monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/category/Legal+Intelligence+Assistants monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/category/LLM+Evaluation monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/tag/AI+Challenges monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/tag/LLM+Implementation monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/tag/Foundation+Data monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/tag/Legal+Tech monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/tag/AI+Implementation monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/tag/AI+Accuracy monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/tag/LLM+Limitations monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/tag/AI+Risks monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/tag/Context+Awareness monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/tag/Legal+AI monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/tag/Context+Retrieval monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/tag/Structured+Data monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/tag/LLM+Performance monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/tag/Sycophancy monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/tag/Mitigation+Strategies monthly 0.5 https://www.simplifylegalintelligence.com/legal-robots/tag/Simplify+Legal+Intelligence monthly 0.5 https://www.simplifylegalintelligence.com/about daily 0.75 2025-04-15 https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/c78d0e1f-f7e1-4c07-b32b-7069e9b2c3d4/012325_Jace_KathyTran_IMG_1348%28E%29.jpg https://www.simplifylegalintelligence.com/foundation daily 0.75 2025-05-02 https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/bbf823a3-406e-4a41-add1-a513417cf79d/SLI+Data+Structure+%28no+bg%29+%284-2-25%29.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/c44cdae3-c539-4bbd-a24e-6735267af55b/SLI+-+Foundation+Data+Image+%283-20-25%29v2.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/be3bb18a-cc08-4c25-8da5-e2fd77404f6a/SLI+Data+Structure+3+%283-22-25%29.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/c7fe7f6b-c49c-45af-9f58-1b194e4b59e9/Messy+Data+-+March+20%2C+2025+-+9_00AM.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/8463c32f-ac73-4622-b383-ad1048b6c0cb/SLI+-+Eval+Image+%283-24-25%29.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/c44d817d-2344-4b9e-a6c8-5e33908eb0e8/SLI+-+Single+Paper+Image+%283-27-25%29.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/be3bb18a-cc08-4c25-8da5-e2fd77404f6a/SLI+Data+Structure+3+%283-22-25%29.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/c7fe7f6b-c49c-45af-9f58-1b194e4b59e9/Messy+Data+-+March+20%2C+2025+-+9_00AM.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/da7c50fd-44ca-4b74-8046-df1192cb98d3/SLI+-+Funnel+Image+%283-22-25%29.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/8463c32f-ac73-4622-b383-ad1048b6c0cb/SLI+-+Eval+Image+%283-24-25%29.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/f7d9218e-4e9f-4d27-9ba9-7c8d89212cd6/SLI+-+Orchestration+Image+%283-22-25%29.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/ecf5e5a7-c86b-4ea4-8e27-ef21d8f146be/Scribble+Logo+%28black+background%29v4+%283-25-25%29.png https://www.simplifylegalintelligence.com/home daily 1.0 2025-04-15 https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/c44cdae3-c539-4bbd-a24e-6735267af55b/SLI+-+Foundation+Data+Image+%283-20-25%29v2.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/4535d8e1-1dae-4583-af11-0cc976dcb96c/Tree+1.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/1e40f66f-79d6-4ba5-ae93-d08bf712c6cd/Tree+2.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/bbffa04c-366a-4c8c-96b7-c5eade3c741b/Tree+3.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/0c7628ed-4935-42c9-a411-a853986f3325/SLI+-+Home+Page+Visual+%284-15-25%29.png https://www.simplifylegalintelligence.com/lia daily 0.75 2025-04-21 https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/ecf5e5a7-c86b-4ea4-8e27-ef21d8f146be/Scribble+Logo+%28black+background%29v4+%283-25-25%29.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/53bffa06-5648-4849-86f3-e3a2b3904f3b/SLI+-+LIAs+Image+%283-25-25%29.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/864f7c4e-c7a5-48ba-91b7-18479bd8f854/SLI+-+Scribble+%28clean+with+icon+and+no+bg%29+%284-1-25%29v2.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/97e46c52-de84-492b-962c-d059d3418618/SLI+-+Bob+%28clean+with+icon+and+no+bg%29+%284-1-25%29.jpg https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/7e538dee-d47d-4016-8885-359c4470dd08/SLI+-+Barri+%28clean+with+icon+and+no+bg%29+%284-2-25%29.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/9cf838d9-6cb5-4b59-ae7c-6d9831912989/SLI+-+Sam+%28clean+with+icon+and+no+bg%29+%284-1-25%29v2.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/f7d9218e-4e9f-4d27-9ba9-7c8d89212cd6/SLI+-+Orchestration+Image+%283-22-25%29.png https://www.simplifylegalintelligence.com/workflow-orchestration daily 0.75 2025-04-15 https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/a9c9ed83-971a-4244-b7f7-8ac93603e700/SLI+-+Orchestration+Image+%28no+bg%29%284-1-25%29.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/ed2636ad-ee36-408a-a5d0-d5c8d994338f/SLI+-+Value+Pyramid+Image+%284-1-25%29v2.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/47d4bd61-bcdb-4ec2-bf0b-23a24de4ebf0/SLI+-+Workflow+Orchestration+Image+%283-31-25%29.png https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/dc1b1d88-dd20-4afe-8f55-2489a9fdd168/SLI+-+Workflow+Modes+Image+%284-2-25%29.png https://www.simplifylegalintelligence.com/scheduling-confirmation daily 0.75 2025-04-15 https://www.simplifylegalintelligence.com/faq daily 0.75 2025-04-30 https://images.squarespace-cdn.com/content/v1/67af3fa35ced7b4f1af37db5/f403ce48-257d-4ec3-a04c-c8c9dfd3825e/SLI+-+Bulb+icon+%28no+bg%29+%284-2-25%29.png https://www.simplifylegalintelligence.com/privacy daily 0.75 2025-04-25 https://www.simplifylegalintelligence.com/supported-foundation-models daily 0.75 2025-04-15