Anthropic S Claude Ai Can Now Control Your Computer Without Any Help

In the official release post for the feature, Anthropic gives us a good look at how the feature works. We see Sam Ringer, an Anthropic Researcher, testing out the tool. He prompts Claude to fill out a request form by extracting information from the spreadsheet and online portal that it’s seeing on the screen.
We then see the tool get to work by screenshotting the information on the screen. Additionally, we see the tool search from an online portal (as directed) to look for information that was unavailable on the spreadsheet. Side-by-side, Claude opened up the form and started filling it up.
Anthropic notes that Claude 3.5 Sonnet scored 14.9% in the screenshot-only category which is “notably better than the next-best AI system’s score of 7.8%.” Additionally, Anthropic states that, “when afforded more steps to complete the task, Claude scored 22.0%.”
Additionally, Anthropic has also released a new Claude 3.5 Haiku smaller model which “improves across every skill set and surpasses even Claude 3 Opus.” Claude 3.5 Haiku will be made available on the API later this month.
Teaching general computer skills to a computer certainly sounds interesting, and something I’m looking forward to. I mean, it has the potential to save a lot of time by automating tasks. What do you think about Claude AI’s computer use feature? Drop your thoughts in the comments down below!