# ollama: Make tool call image output accessible to the model \(\#64121\) · gitcafe/zed [View on GitCafe](https://git.cafe/gitcafe/zed/commit/fe1dd2d375f1085e957a5927b7a3a5c294aa989f) Repository: [gitcafe/zed](https://git.cafe/gitcafe/zed) Visibility: public Requested revision: fe1dd2d375f1085e957a5927b7a3a5c294aa989f Requested commit: fe1dd2d375f1085e957a5927b7a3a5c294aa989f Commit: fe1dd2d375f1085e957a5927b7a3a5c294aa989f Tree: e609b12ca7d5a82eecbd415b4cdd59f030765c7a Author: marius851000 Committer: GitHub ## Message ```` ollama: Make tool call image output accessible to the model (#64121) # Objective - Tool call can output image. Ollama did not support that, and dropped them when generating the Ollama request. The images are now properly added. - Fixes #64054 ## Solution - Convert the data from Zed’s internal structure to Ollama’s structure - Should avoid allocation if there is no images (the most common path) (untested). ## Testing - Tested with qwen3.6:35b via Ollama. It can read and describe image, with query such as "Describe zébrette_le_matin_22_novembre_1913.jpg" (the file not being directly embedded). The agent use the read_file tool. ## Self-Review Checklist: - [x] I've reviewed my own diff for quality, security, and reliability - [x] Unsafe blocks (if any) have justifying comments (there are none) - [x] The content adheres to Zed's UI standards ([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist) and [icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md) guidelines) (no UI changes) - [ ] Tests cover the new/changed behavior - [x] Performance impact has been considered and is acceptable ## Showcase
Transcription of a test query ## User Describe zébrette_le_matin_22_novembre_1913.jpg ## Assistant The user is asking me to describe an image file: "zébrètte_le_matin_22_novembre_1913.jpg". This appears to be a newspaper/magazine cover or page from "Le Matin" dated November 22, 1913. I need to read this image file to see what's in it. Let me find it first. Let me locate and examine that image file first. **Tool Call: No matches** Status: Completed ```json { "offset": 0, "current_matches_page": [], "all_matches_len": 0 } ``` ## Assistant Let me try different variations of the filename. **Tool Call: 1 match** Status: Completed file:///home/marius/sec/a/z%C3%A9brette_le_matin_22_novembre_1913.jpg ## Assistant Found it. Let me read the image file. **Tool Call: Read file `zébrette_le_matin_22_novembre_1913.jpg`** Status: Completed `Image` ## Assistant Now I can describe the image. It's a black-and-white illustration from what appears to be a French newspaper ("Le Matin" dated November 22, 1913). The image depicts an elegantly dressed woman descending a staircase. She is wearing: - An elaborate headdress with what looks like ostrich feathers or plumes - A decorative crown or tiara - Long earrings - Multiple strands of pearls around her neck - A flowing, ornate dress with fur trim (a stole/cape) - A long necklace The style is Art Nouveau/Art Déco. She appears to be a society figure attending some gala or fancy dress event. The illustration has a halftone dot pattern typical of newspaper printing from that era. I'm wondering what "zébrètte" refers to—it might be the name of the person depicted, the caption associated with this illustration, or perhaps a specific fashion item. ## Description: *Zébrètte* (Le Matin, 22 novembre 1913) This is a black-and-white newspaper halftone illustration from the French daily **Le Matin**, dated November 22, 1913. It depicts an elegantly dressed woman descending a staircase — likely covering a high-society or gala event. ### Key details: - **Headdress**: The most striking feature is an elaborate plumed hat with large ostrich-style feathers rising above her head, paired with a decorative crown/tiara. - **Attire**: She wears a flowing gown adorned with ornate patterns, draped under a luxurious fur-trimmed stole (or capelet). Her dress has intricate embellishments on the bodice and legs of the skirt. - **Accessories**: Multiple strands of pearls around her neck, long dangling earrings, and an additional long necklace chain add to the opulent look. - **Pose & setting**: She steps down a staircase with a swirling Art Nouveau–style railing behind her, suggesting she has just arrived at or is leaving a grand event (a ball, premiere, or society gathering). - **Artistic style**: Rendered in a sketchy, expressive line-drawing style typical of early 20th-century French illustration. The halftone dot pattern reveals it was printed in a newspaper. The caption title *"Zébrètte"* likely refers to either the nickname of the woman depicted or a costume/carnival reference (suggesting "striped" or zebra-print), though without the surrounding text from the original page, that detail remains ambiguous. --- The image in question, I had used as an illustration for one my blog post, as said by the AI from the Le Matin of the 22 november 1913, via Gallica (https://mariusdavid.fr/blog/z%C3%A9bressez%C3%A9brellez%C3%A9brette/) image
--- Release Notes: - Fixed Ollama not being able to access images from tool call --------- Co-authored-by: Bennet Bo Fenner ```` ## Parents - [95c0d74be964fa216acdfa2cbd1f247862576216](https://git.cafe/gitcafe/zed/commit/95c0d74be964fa216acdfa2cbd1f247862576216?format=markdown) [Source at this commit](https://git.cafe/gitcafe/zed/tree/fe1dd2d375f1085e957a5927b7a3a5c294aa989f?format=markdown)