AI Passes the Bar Exam

The latest release of GPT (GPT-4) now does quite well on many scholastic and professional tests, especially compared to its predecessor (GPT-3.5) from just a few months ago:

That represents nearly the 90th percentile on the bar exam. A newly minted lawyer, even one who is in the 90th percentile on the bar exam is still not quite useful. So the question is: will these Large Language Models like GPT continue on their current rapid trajectory of improvement and become critically important tools for lawyers and other professionals? I decided to ask GPT-4 directly for its opinion on the matter. Here is my query:

Large Language Models such as GPT-4 have made impressive improvements in their abilities to score well on various scholastic and professional tests such as the bar exam. Do you think these improvement trends will continue or is GPT-4 near the limit of Large Language Models' capabilities?

And here is GPT-4's response:

It is difficult to predict with certainty the future of large language models like GPT-4, but recent trends suggest that there is still potential for improvement in their capabilities. The development of AI and machine learning is a fast-paced field, and advancements continue to be made in the areas of model architecture, training methods, and data availability.

As these models become more powerful and sophisticated, it is possible that they will continue to perform better on scholastic and professional tests. However, several factors may influence the rate of improvement and the ultimate capabilities of these models:

1. Diminishing returns: As models become larger and more complex, the improvements gained from increasing their size may start to diminish. Researchers may need to explore alternative approaches, such as more efficient architectures or better optimization techniques, to achieve significant improvements.

2. Computational resources: Training large language models requires significant computational power and resources, which can be expensive and environmentally impactful. This may impose practical limits on the size and complexity of future models, leading researchers to focus on more efficient training methods or smaller, specialized models.

3. Understanding and reasoning: While large language models like GPT-4 can generate coherent and contextually appropriate responses, they may still struggle with deep understanding and reasoning. Improving these capabilities will likely require advancements in AI research beyond simply increasing model size and may involve incorporating other techniques, such as symbolic reasoning or reinforcement learning.

4. Ethics and safety: As AI models become more powerful, concerns about their ethical use and safety increase. OpenAI and other organizations are actively researching ways to mitigate risks, such as biases in the training data and potential malicious use. The development of future models will need to balance capabilities with these ethical and safety concerns.

In summary, while it is likely that large language models will continue to improve, the extent and rate of these improvements will depend on various factors, including advancements in AI research, computational resources, and ethical considerations.

That's a more extensive answer than I would've given as I wouldn't've considered factors such as environmental impact or safety. Other than that, I agree with its assessment and it was certainly easier to let it write the answer for me. It may not really be useful as a lawyer yet, but it's certainly pretty good as a blogging assistant!!!