KlikoAI

From NURDspace
Revision as of 19:49, 14 April 2026 by Melan (talk | contribs)
Jump to navigation Jump to search
KlikoAI
KlikoAI.jpg
Participants Melan
Skills
Status Active
Niche Smart stuff
Purpose Home Automation
Tool No
Location
Cost
Tool category

KlikoAI Property "Tool Image" (as page type) with input value "File:{{{Picture}}}" contains invalid characters or is incomplete and therefore can cause unexpected results during a query or annotation process. {{{Picture}}} {{#if:No | [[Tool Owner::{{{ProjectParticipants}}} | }} {{#if:No | [[Tool Cost::{{{Cost}}} | }}


KlikoAI detects whether the paper and waste containers at NURDspace are inside or outside. Previous approaches using distance sensors and BLE sensors proved unreliable, so the project switched to AI-based vision detection instead.

KlikoAI uses mistral3:14b via Ollama as the backend. Before that, moondream was used with inference running inside KlikoAI itself, but it struggled with certain container configurations. mistral3:14b showed better performance on this task.

It runs on gpu.vm.nurd.space (via Docker, in ~/klikoai) on an Nvidia GeForce RTX 3080.

KlikoAI uses the Home Assistant calendar integration to detect trash pick-up days and switches to a more frequent scan schedule on those days, falling back to once every 6 hours otherwise. There is also logic to detect significant changes in frame, though this has not yet been tested.

State is published as an MQTT sensor to Home Assistant, which allows the "PUT THE TRASH OUTSIDE" notification to be muted once the correct container has been detected outside. There is also IRC integration via the !klikos and !klikoupdate commands (the latter triggers a manual scan).

Previous Iteration

The original plan was to train a YOLO model on an existing garbage container dataset. Due to the limited camera mounting options at the space, detection results were poor, which led to switching to an LLM vision model.

Dataset

The bins at NURDspace have a blue lid and a yellow lid on an otherwise green body. Finding training images with those specific lid colours turned out to be harder than expected.

YOLO returns bounding boxes, so one option would have been to inspect just the top portion of the detected box and check the colour range, or determine bin position based on where it appears in frame.

Since the default YOLO models do not support garbage bin detection, a custom model was trained on a 5090 using this dataset from Roboflow.