alt.hn

I'm not in the legal field, but can someone explain that further? I would have expected that all case law is public access. Not necessarily easy access, but when a judge writes an opinion, why on Earth would that opinion be gated behind a corporation? What am I missing?

by hn_throwaway_99

Theoretically speaking if someone scraped all of it and added it to something like this open source Mike project would that then be a much better tool for lawyers?

by r0fl

I'd not use generative AI for anything but a cursory check anyway⁰. Even if it is trained on clean up-to-date data rather than all the wrong information that is out there, it could still give a wrong answer and I have no leg to stand on if I rely upon it. At least if I pay a human and they trust the LLM too much, I'll hopefully have some call to pursue them for giving bad advice when it bites me.

--------

[0] Or at all… But even if I wasn't someone actively avoiding LLMs, the point would still stand

by dspillett

And this is not likely to change in the future, as the legal market is so small and niche that the leading makers of LLMs have put legal analysis near the bottom of their list of priorities in terms of improving model performance. There is very little if any effort by the major LLM companies to curate sources of additional high-quality legal training data or fine-tune their models to improve performance on legal tasks. Law is also a field with very low tolerance for error, where tiny mistakes can have big consequences, and getting the models to perform well under these constraints would require a lot of investment without a sufficient payoff.

The true reason big firms are buying Harvey and Legora subscriptions is simply to use an LLM for LLM-type tasks, like document review, spotting issues in user-provided documents, and other things that LLMs do well. True, services like Harvey and Legora have lots of cool templates and features for legal work, but you will find that most of the people who use these services in these firms use them much the same way they'd use ChatGPT, Claude, or any other AI chatbot.

The reason law firms can't just use ChatGPT or Claude is that they can't allow confidential or privileged client data (such as documents provided through the prompts) to be stored and hosted on a third party service like ChatGPT or Claude, as these companies may have to turn over client data in response to subpoenas, and depending on the type of LLM account you have, these companies could use your user prompts to train future models thus risking leakage of client data to third parties and potential privilege waiver.

Services like Harvey and Legora solve this problem by accessing the LLMs through APIs, and all client data, prompts/responses, etc., are stored encrypted on Harvey or Legora servers and protected by keys held by the customer. For many law firms, this is 95% of what they're paying for.

The big challenge "Mike" presents to services like Harvey and Legora is that it exposes how little additional value they offer over ChatGPT or Claude, for the vast majority of law firms. A system like "Mike" can provide the same security benefit at basically $0 cost, and can be hosted on the law firm's own internal servers. This is going to put a lot of pricing pressure on services like Harvey and Legora; law firms are notoriously cheap when it comes to IT and software spend and will switch quickly if cheaper alternatives arise. This confirms that Harvey and Legora are going to have to sell their services based on the value they add to lawyer productivity, and not just on being a protected wrapper around GPT or Claude.

by VoidCoefficient

Also, an interesting example: in English litigation (where, broadly, loser pays unlike America where each side pays), maximising billable hours is not always a viable strategy for anybody if those costs aren't recoverable on success. Someone involved in large-scale commercial litigation involving disclosure of millions of documents who doesn't use algorithmic document classification (now pretty broadly accepted as normal) potentially runs the risk of a judge determining that the costs of going through all the documents by hand isn't recoverable. Insurers/litigation funders aren't going to want to risk padding the costs so much that the judge prevents them from recovering their stake in the litigation.

Customers using their own LLMs: yep, they might do that. I think the pitch from the legal LLM providers is "we've got legally trained people doing RLHF to make it more accurate" mixed in with "also we've got a partnership with Lexis/Westlaw/etc. so we can do legal research that's better than what's on the open web", with a little bit of "if you get sued for professional negligence, 'I used the legal AI thing that's built into Westlaw' is gonna be more convincing to a judge and jury (and your insurance company) than 'I used ChatGPT, yes, like the app you've got on your phone'...".

by lawtalkinghuman

We don't have paralegals/attorneys handle cases from beginning to end. We have different positions handle different tasks. One person may only do scheduling, another does discovery, another handles reviewing releases.

For us, adopting tech to make us more efficient is a priority. Our setup is a bit unique, but I can see PI and collection firms adopting tech similar to this.

by yeehawtypebeat

But that only applies for clients using the chatbot. If a lawyer is using the LLM it is definitely protected. No different if a lawyer searches something on Google or Lexis Nexis. The search itself is protected. I guess you could debate metadata but the content surely is protected.

by robertritz

I don’t actually care that much about the work having been hard - I care about the result being good.

by solarkraft