Artificial Intelligence (AI) is still very much the buzzword of the moment within the legal profession. A recent study conducted by LexisNexis found that 82% of lawyers are currently using or plan to use AI in their day-to-day work. This was up from only 39% in July 2023.

In September last year, we conducted an interview with the AI Chatbot, ChatGPT to see how much it really knew about our business. The results were somewhat uninspiring. It’s not that the answers to the questions we asked were wrong; they were just incomplete (and lacked a bit of personality). In the service industry, many of us have been warned that AI will imminently replace us, but our interview made clear to us that it wasn’t quite time for an early retirement.

So did we write AI off? No – of course we didn’t. Instead, we wanted to give it a second chance and see how far it has come in the last year and a bit. However, in the interests of fairness, this time we asked the same questions to both ChatGPT and Copilot to see how two different AI tools compared to each other and also to reality.

The answers for both chatbots have been copied and pasted in their entirety. We have not embellished them further and we have not added sources ourselves.

The Interview

Who are Pinsent Masons Vario?

Copilot

Pinsent Masons Vario is a division of the international law firm Pinsent Masons, offering a range of alternative legal services. Vario provides flexible legal resourcing, managed legal services, legal project management, and legal technology consultancy¹².

Here are some of the key services they offer:

• Flexible Legal Resourcing: They provide freelance lawyers, paralegals, and other legal professionals to support clients on a temporary basis¹².
• Managed Legal Services: This service helps clients manage routine legal tasks, allowing their in-house teams to focus on more strategic work².
• Legal Project Management: Vario offers project management services to ensure legal projects are delivered efficiently and effectively¹.
• Legal Technology Consultancy: They help clients implement legal technology solutions to improve efficiency and reduce costs².

Vario operates globally, supporting clients across various jurisdictions, including the UK, Ireland, Germany, and the Asia-Pacific region².

ChatGPT

Pinsent Masons Vario is the flexible resourcing and consulting arm of the international law firm Pinsent Masons. It focuses on providing innovative legal services and solutions, leveraging technology and a diverse talent pool to meet clients' needs. Vario offers a range of services, including access to freelance lawyers, legal project management, and other tailored support options, allowing clients to adapt to changing demands and optimize their legal operations. The aim is to provide a more agile and cost-effective approach to legal services compared to traditional law firm models.

Vario’s Verdict

Both chatbots correctly identified 4 out of our 5 service lines, with only Company Secretarial Services (Co Sec) being missed in each case. This service has traditionally fallen within our Managed Legal Services umbrella but has recently become a distinct service offering of its own. However, we can forgive the omission on this occasion given that there is not yet a separate Co Sec page on our website.

Despite this, the strongest overall answer was the Copilot answer for 3 primary reasons:

1. It identifies Vario as an ALSP for Pinsent Masons as opposed to just a provision of flexible resourcing et al.
2. Its presentation and answer structure was far clearer.
3. The primary sources for the information given were cited, allowing us to see where their answer was pulled from.

What sectors does Pinsent Masons Vario operate in?

Copilot

Pinsent Masons Vario operates in a variety of sectors, leveraging their expertise to provide tailored legal services. Some of the key sectors they work in include:

• Energy and Natural Resources: Supporting clients in the energy sector, including renewable energy projects¹.
• Construction: Providing legal services for construction projects, from planning to completion¹.
• Real Estate: Assisting with real estate transactions, development, and management¹.
• Financial Services: Offering legal support to financial institutions and services¹.

These sectors benefit from Vario’s flexible legal resourcing, managed legal services, legal project management, and legal technology consultancy¹.

ChatGPT

Pinsent Masons Vario operates across several sectors, providing flexible legal solutions tailored to various industries. Key sectors include:

1. Construction and Engineering: Offering support in contracts, disputes, and project management.
2. Energy: Providing legal expertise related to renewable energy, oil and gas, and utilities.
3. Financial Services: Addressing regulatory, compliance, and transactional needs.
4. Technology: Supporting clients with issues related to IT, data protection, and intellectual property.
5. Real Estate: Assisting with property transactions, development, and leasing matters.
6. Telecommunications: Offering guidance on regulatory frameworks and commercial agreements.
7. Life Sciences and Healthcare: Providing legal support in areas such as pharmaceuticals, biotechnology, and healthcare regulations.

Vario's flexibility allows it to adapt its services to meet the specific requirements of each sector.

Vario’s Verdict

Both answers appear partially correct, but neither concisely identifies the sectors that Pinsent Masons operates within. As a Practice Group within the wider firm, Vario’s sectors of focus are primarily the same as Pinsent Masons’, these being:

1. Energy
2. Real Estate
3. Financial Services
4. Infrastructure
5. Technology, Science & Industry

An element of reading between the lines and engaging in informed research would have revealed this, but both chatbots seemed to aim for a largely scatter graph approach. It does however highlight that maybe we could spell this out a little more explicitly in the future.

Are Pinsent Masons Vario’s clients typically corporate or personal?

Copilot

Pinsent Masons Vario primarily serves corporate clients. Their client base includes a wide range of organizations, from large multinational companies to public sector organizations¹. They provide services across various sectors such as banking, technology, media, telecommunications, and infrastructure

ChatGPT

Pinsent Masons Vario primarily serves corporate clients rather than individual consumers. Its focus is on providing flexible legal services to businesses across various sectors, helping them with complex legal needs, compliance, and project management. Vario collaborates with organizations of different sizes, including multinational corporations, SMEs, and startups, to deliver tailored legal solutions that align with their operational demands.

Vario’s Verdict

Both chatbots were correct in their answers here, and even gave some nice examples of the types of organisations we work with. Bonus points go to Copilot again for citing its source, but overall, both of these answers can be considered accurate.

What is the biggest challenge faced by in-house lawyers and how could Pinsent Mason Vario help with a solution?

Copilot

In-house lawyers face several significant challenges, but one of the biggest is managing heavy workloads with limited resources. This often includes handling a high volume of contracts, ensuring compliance with ever-changing regulations, and managing risk while maintaining efficiency¹²³.

Pinsent Masons Vario offers several solutions to help in-house legal teams tackle these challenges:

1. Flexible Legal Support: Vario provides access to a pool of experienced freelance lawyers who can be brought in to support specific projects, cover staff absences, or manage peak workloads⁴⁵. This flexibility helps in-house teams manage their workload more effectively without the need for permanent hires.
2. Legal Project Management: Vario’s legal project management services ensure that legal tasks are completed efficiently and on time. This allows in-house lawyers to focus on providing high-quality legal advice rather than getting bogged down in administrative tasks⁴.
3. Legal Technology Consultancy: Vario helps in-house teams implement legal technology solutions that streamline processes, such as contract management and compliance tracking⁴. This can significantly reduce the time and effort required to manage routine legal tasks.
4. Managed Legal Services: Vario offers managed legal services that take over certain legal functions, allowing in-house teams to focus on more strategic activities⁴. This can include everything from routine contract reviews to more complex legal matters.

By leveraging these services, in-house legal teams can improve their efficiency, reduce stress, and better manage their workloads.

ChatGPT

One of the biggest challenges faced by in-house lawyers is managing workload and resource constraints, particularly during peak periods or when dealing with complex legal issues. In-house teams often face pressure to deliver high-quality legal support with limited personnel and budget.

Pinsent Masons Vario can help by providing flexible legal resourcing solutions. They can offer skilled interim lawyers or specialists on a project basis, allowing in-house teams to scale resources up or down as needed. This approach not only alleviates immediate workload pressures but also ensures access to expertise without the long-term commitment of hiring additional staff. Furthermore, Vario’s model allows in-house counsel to focus on strategic initiatives rather than being bogged down by routine tasks.

Vario’s Verdict

Both chatbots said that the biggest issue was managing workload in the face of limited resources. This is certainly something that we see when talking to our clients, but whether it’s the biggest challenge facing in-house lawyers is up for debate. Do you think that this is correct, or have the chatbots missed something here?

Conclusion

So the key question is, has AI improved over the last year? In short, it has.

The answers given across the board were more sophisticated to those given for last year’s questions. Whilst it’s important to note that some of this year’s questions were different to last year’s ones, the level of detail provided was better and more accurate overall. It was particularly impressive to see Copilot citing its primary sources for information without us needing to ask for it.

Would we trust a chatbot to take over from our fee earners? No. However, the current industry trends suggest that more and more legal practitioners are utilising AI as a means of becoming more efficient at their jobs. On the basis of this interview, we understand why that might be the case.