Medal, a startup company that is better known for its video game clipping product, just announced that it has acquired $13 million at a valuation of $333 million from several investors, among them are Horizons Ventures, OMERS Ventures, Peak6, and Arcadia Investment Partners.

To this, the company added a new cross-platform desktop app of contextual AI assistant, referred to as Highlight, which captures the content in a user's screen and lets them ask questions to a large language model based on that context.

According to Henry Gladwyn, the partner at OMERS Ventures, talking to TechCrunch over a call, "the venture firm saw the chance in applying Medal's core technology, onto LLMs".

So, fundamentally, the core Medal tech is all about understanding what is going on on somebody's device-the video, the audio, and the stuff going on around it. That was first used for clipping. Now, the company takes that tech and applies that to LLMs for prompting, which is a pretty clever use, " said Gladwyn.

Gladwyn said, he regarded medal as a product for recording best bits of virtual life rather than being just a gaming company. And the Highlight app is an organic extension of that paradigm.

How does Highlight work?

For years, companies have been trying to build a useful assistant for users using on-screen information. Google has been trying since Google Now, through Google Assistant, and now Gemini. The company's foray into this space came last month at its Worldwide Developers Conference (WWDC), where it announced Apple Intelligence and its ability to understand on-screen contextual information. Microsoft is also looking to the generative AI side with Windows Recall, a feature that helps users find content they have viewed in the past — after announcing this feature, Microsoft decided to delay its launch of Recall.

Highlight is attempting to make this a reality for desktops. In the current version, the app exists as a floating button on your desktop. When hovered over the icon, it captures the content of the screen and passes it as context to various models. You can choose to ask other questions using different tools such as ChatGPT, Anthropic's Claude, Perplexity, and more.

It already prepopulates some of the questions so you're given a chance to get underway. Capture happens locally, and no content is stored in the app. The company is building its own chat assistant, which might be less capable for some tasks but should eventually run locally on your device.

Aside from that, which is being rendered on screen, you can also pass a document and system audio memory as context to Highlight. To be more specific, the company is building a local transcription app for meetings akin to tools such as Granola, Limitless, and Krisp.
Building Highlight
Medal's one cofounder, Pim de Witte, over a call told TechCrunch that the company started brainstorming last year as to what ways they could apply Medal.tv's clip capture technology along with AI.

"We know that recoding activities will be important for operating systems, and we have seen some large tech companies making such moves. We want to give users an open platform to connect with assistants, models, and interfaces," de Witte said.

And he said that he wanted to create an app with AI that people of all ages, even not knowing about this technology, would be able to use it. And that's why with the capturing, the app gives out the contextual prompt suggestions.

The company is also building an open platform for developers to deploy their own apps on the Highlight platform. This is, of course, somewhat the equivalent of Raycast's launcher application for Mac, which allows you to install extensions that developers create, but one important difference is that Highlight is both on Mac and Windows.

Omers Ventures told Gladwyn Highlights is an advantage as independent and has no interest in drawing you into one ecosystem.

The way going
Medal.tv being successful product the company wants to keep supporting it adding new features, however, the company will give some staff to work on the highlight.

Highlights is free for now but de Witte wants to monetize it with the app store model. A premium subscription will offer access to some of its own apps and features like local-first models.
The company also offers grants worth up to $30,000 to developers, who'll also be granted access to Highlight's team to develop the app ecosystem.