Answers questions about your photos, screenshots and videos, and describes a clip moment by moment

You give it a picture, a few pictures or a video, then ask a question in plain words. It answers in text. Point it at a photo of a broken appliance and ask what the warning light means. Drop in three screenshots and ask which plan is cheaper. Give it a screen recording and ask for a step by step list of what happened, including the text that appeared on screen.

Video is where it stands out for its size. It samples frames across the whole clip and describes the scenes in order, so you get a timeline instead of a guess based on the first frame. It also reads text in images well enough to pull lines out of a scanned page or a menu photo.

The grounding detail: it has 1 billion parameters, small enough that the makers show it running on an iPhone in an unedited recording, and their tests put it clearly ahead of Qwen's similar sized model. It is released under the Apache 2.0 license, so you can build it into a product.

The official demo Space runs it in your browser with nothing to install. First open the model menu and switch it from the larger 35B version to MiniCPM-V 4.7 1B, since the demo starts on the big one. Then upload a short video from your phone and ask: "Describe this video in order: what happens and what text appears on screen." Compare its timeline with what you remember filming. After that, try a screenshot of a receipt and ask for the total. To run it yourself, it is on Ollama under openbmb/minicpm-v4.7, and the MiniCPM-V Apps project has the source for iPhone and Android apps.

It is a 1 billion parameter model, so expect it to miss small print on dense pages and fine detail in long videos: it looks at up to 128 frames per clip, which for a ten minute video is about one frame every five seconds. The demo's code also keeps copies of uploaded files in its logs by default, so do not upload anything private there; run it locally for that.

• Get a written, step by step summary of a screen recording for a bug report.
• Ask what is happening in a long phone video without scrubbing through it.
• Pull the text and numbers out of photographed receipts, menus or slides.
• Compare two or three product screenshots side by side in one question.
• Build a photo or video assistant into an app that works without a cloud service.

Try the demo

View model page

Read this issue on neuralexpedition.com →