Telescope integrates Claude 3 into its model-agnostic AI platform for finance

We are excited to announce the integration of Anthropic’s Claude 3 model family, launched just yesterday on March 6, 2024, into our technology stack at Telescope. As a leading provider of generative intelligence solutions for finance, this development enhances our ability to deliver advanced AI-driven tools for brokerages, publishers, and fund managers, while maintaining our commitment to a model-agnostic approach.

The Claude 3 model family, which includes Haiku, Sonnet, and Opus, sets new industry benchmarks across a wide range of cognitive tasks. Each successive model offers increasingly powerful performance, allowing our users to select the optimal balance of intelligence, speed, and cost for their specific application.

Impressive performance metrics

Anthropic’s Claude 3 models have demonstrated remarkable capabilities in various benchmarks, outperforming many of their peers. Opus, the most advanced model in the family, has shown near-human levels of comprehension and fluency on complex tasks, leading the frontier of general intelligence.

According to Anthropic’s benchmarks, Claude 3 Opus achieves:

• 86.8% accuracy on the MMLU benchmark, which tests undergraduate-level expert knowledge across various domains
• 50.4% accuracy on the GPQA benchmark, which evaluates graduate-level expert reasoning
• 95.0% accuracy on the GSM8K benchmark, which assesses basic mathematics skills

These impressive results, highlight the potential of Claude 3 to revolutionize AI applications in finance and beyond.

Benchmarking at Telescope

Our team at Telescope is actively testing and benchmarking the Claude 3 model within our platform, specifically Sonnet. Our focus is on finance-specific tasks such as investment portfolio generation, investment thesis writing, and compliance checks. The initial results are promising, with Claude 3 Sonnet demonstrating superior performance compared to previous models on face value.

”We’re thrilled to integrate Claude 3 into our platform and explore its capabilities in the finance domain,” said Luc Pettett, our Founder at Telescope. “The development of Anthropic and the benchmarking results against GPT-4 are very impressive. We believe this model upgrade has the potential to significantly enhance our solutions.”

Fewer refusals: A welcome improvement

One of the key factors that led us to switch from Claude 2.0 to GPT-4 for many of our tasks was the issue of unnecessary refusals. These refusals often suggested a lack of contextual understanding, which could hinder the performance of our AI-driven solutions.

According to Anthropic, the new models – Opus, Sonnet, and Haiku – are considerably less likely to refuse to answer prompts that border on the system’s guardrails compared to previous generations of models. For Telescope, this was a considerable issue. The Anthropic model family demonstrates a more nuanced understanding of requests, recognizes real harm, and refuses to answer harmless prompts much less often. This improvement in contextual understanding and reduced unnecessary refusals has rekindled our interest in exploring Claude 3’s capabilities within our platform.

We are beginning our tests across Claude 3 models in action to assess their performance in handling complex financial tasks. If the models live up to Anthropic’s claims, this could mark a significant step forward in the usability and reliability of AI-driven tools in the finance industry. We look forward to sharing our findings with our clients and the broader finance community as we continue to benchmark and integrate cutting-edge AI models into our platform.

Enhanced resistance to jailbreaks and misuse

Another compelling feature of the new model family is its enhanced resistance to jailbreaks and misuse. In the rapidly evolving landscape of AI, ensuring the security and integrity of AI-driven solutions is of utmost importance, especially in the finance industry where trust and reliability are paramount.

Anthropic claims that the Claude 3 models are 10 times more resistant to jailbreaks and misuse compared to their predecessors. This significant improvement in security is a testament to Anthropic’s commitment to developing responsible and trustworthy AI systems.

Our commitment to model-agnostic AI

As a model-agnostic platform, we at Telescope continuously evaluate and incorporate cutting-edge AI models to ensure our clients have access to the best tools for their specific needs.

”Our model-agnostic approach allows us to rapidly benchmark the performance of various AI models and select the most appropriate one for each use case,” explained Pettett. “By integrating Claude 3 into our platform, we can offer our clients a higher quality product and outcomes.”

As we continue to push the boundaries of generative intelligence in finance, the integration of Claude 3, just one day after its launch, marks a significant milestone in our journey. With its impressive performance metrics and our commitment to model-agnostic AI, this collaboration is poised to drive innovation and empower financial institutions to make more informed decisions.