Computer Vision news and engineering summaries

Waymo has introduced the Waymo World Model, a pioneering generative AI system designed for hyper-realistic autonomous driving simulation. Built upon Google DeepMind's Genie 3, the model moves beyond traditional on-road data by leveraging vast pre-trained world knowledge to simulate rare, long-tail scenarios such as extreme weather or unexpected obstacles. The system features high controllability through language prompts, scene layouts, and driving inputs, allowing for 'what-if' counterfactual testing. Crucially, it generates multimodal outputs including both camera imagery and 4D lidar point clouds, providing a comprehensive training environment for the Waymo Driver. This advancement enhances road safety by preparing the vehicle for complex edge cases long before it encounters them in reality, significantly scaling Waymo's ability to deploy across diverse urban environments.

OpenCV 5 is a major modernization of the world's most widely used computer vision library. Key highlights include a completely rewritten, graph-based DNN engine with 80%+ ONNX support, built-in LLM/VLM capabilities, improved hardware acceleration via a new HAL, native support for modern data types, and modernized 3D vision and documentation. OpenCV 5 maintains core API stability while significantly enhancing performance and flexibility.

Google introduces Nano Banana 2, a state-of-the-art image model combining the reasoning of Nano Banana Pro with Flash-level speed. It features advanced world knowledge for infographics, precise text rendering, and improved subject consistency. The model integrates SynthID and C2PA credentials for robust provenance and is rolling out across the Gemini app, Search, and Vertex AI.

This article explores atmospheric scattering in shaders to render realistic skies and planetary atmospheres. It details the step-by-step implementation of Rayleigh and Mie scattering, ozone absorption, and the use of LUTs for performance optimization. The guide explains integrating these effects as post-processing into WebGL scenes to create accurate lighting, sunsets, and volumetric fog.

RuView is an edge-based spatial awareness system that reconstructs human pose, heart rate, and breathing using WiFi and radio signals instead of cameras. Built on Rust, it offers 810x speedup over Python, running on low-cost ESP32 hardware. Key features include privacy-first sensing, through-wall detection, and self-learning models for healthcare, security, and disaster response.

GenCAD is an image-conditional generative model that produces both 3D CAD models and their underlying parametric command sequences. By utilizing transformer encoders, contrastive learning, and diffusion models, GenCAD overcomes limitations of mesh or voxel representations, enabling precise, modifiable CAD generation essential for engineering and manufacturing workflows.

Waymo has released updated safety data illustrating that its autonomous 'Waymo Driver' significantly outperforms human drivers in areas where it operates. Based on over 170 million rider-only miles, data show major reductions in crash rates involving injuries, serious injuries, and airbag deployments, demonstrating superior performance in avoiding collisions with pedestrians, cyclists, and other vehicles.

Waymo has unveiled its 6th-generation Driver, a fully autonomous system featuring streamlined hardware and custom-designed sensing technology. By integrating high-resolution cameras, lidar, and radar with advanced AI, the system reduces costs while enhancing performance in diverse weather. This scalable architecture is designed for high-volume production across multiple vehicle platforms.

A$AP Rocky's music video for 'Helicopter' represents a significant milestone in technology, utilizing dynamic gaussian splatting and volumetric capture on a large scale. Produced by teams including Evercoast and Grin Machine, the project captured human performances using a 56-camera array to generate over 10 terabytes of data. This workflow allowed for radical post-production freedom, enabling the team to relight, recontextualize, and manipulate 3D performances within Houdini and OctaneRender. Unlike AI-generated content, every motion was physically performed and spatially preserved, demonstrating how radiance fields can offer directors unique creative flexibility without sacrificing the authenticity of human movement in high-end music production.

This guide explains how to transform Gaussian Splatting environments into interactive web-based videogames using PlayCanvas. By integrating physics colliders via voxelization, light baking, navigation meshes, and behavior-tree AI, the author demonstrates how to create a fully playable FPS experience that runs in a browser without needing complex geometry or heavy rendering engines.

This gallery showcases high-quality 3D reconstructions using OpenScan technology. Featuring models like the Giant Swallowtail and Ammonite, it highlights the versatility of OpenScan Classic and OpenScan Mini combined with DSLR cameras and photogrammetry software. These textured meshes demonstrate applications in entomology, paleontology, and digital preservation using tools like OpenScanCloud and 3DF Zephyr.

Google introduced Gemini Robotics-ER 1.6, an upgraded model for embodied reasoning that enhances how robots perceive and navigate the physical world. Featuring improved spatial understanding, multi-view processing, and specialized capabilities like instrument reading and success detection, the model enables greater autonomy. It is now available via the Gemini API and Google AI Studio for developer use.

Steam Controller Auto-Charge is an open-source web application using computer vision and WebHID to automate docking a Steam Controller. It leverages OpenCV.js for optical flow tracking and Rust/WASM for object detection, navigating via haptic pulses. The system, built with Vue 3, works across platforms using the Nix package manager and browser-based WebHID API.

A developer created an iOS app combining OpenCV geometry analysis and a fine-tuned YOLOv8 model to automate scoring for shooting targets. By replacing manual brass gauges with on-device computer vision, the project provides precise hole detection and performance analytics, while reflecting on the human desire to innovate and improve traditional processes through technology.

Researchers from the University of California, Santa Cruz, and Johns Hopkins have demonstrated a new vulnerability in autonomous systems called CHAI (Command Hijacking against Embodied AI). This environmental indirect prompt injection attack targets Large Vision Language Models (LVLMs) used in self-driving cars and drones. By placing manipulated road signs with specific text, colors, and fonts, attackers can override an AI's decision-making process. Experiments showed that autonomous vehicles could be tricked into ignoring pedestrians at crosswalks or redirected by signs reading 'Proceed,' while drones were deceived into landing on debris-filled rooftops or following incorrect vehicles, with success rates exceeding 90% in some scenarios.

Chinese electric vehicle giant BYD is integrating high-end LiDAR sensors into its most affordable models, including the Seagull and Dolphin. Previously reserved for premium vehicles priced above $25,000, this expansion marks a significant milestone in bringing advanced driver assistance system technology to the mass market. These budget models are expected to feature a comprehensive sensor suite incorporating 12 cameras, 5 mm wave radars, and ultrasonic sensors. This hardware upgrade enables Navigate on Autopilot functionality for both highways and city streets. While the Australian version, known as the Atto 1, does not yet include this camera suite, the global rollout of these LiDAR-equipped variants signals an intensifying competition in the assisted-driving sector. BYD's strategy leverages its massive production scale to democratize autonomous driving features across its international lineup.

This browser-based Gaussian splat generator utilizes Apple's SHARP model to convert single images into 3D Gaussian splats. Built with React and TypeScript, it uses ONNX Runtime Web for local inference, offering in-page previews and .ply file exports. This experimental tool demonstrates sophisticated client-side 3D reconstruction using WebGPU and WASM, requiring significant memory and proper model asset serving.

IMAGE-BLASTER is an open-source tool that converts single images into 3D environments, meshes, and SFX in minutes. It leverages Claude, World Labs, FAL, and ElevenLabs to generate 3D models, Gaussian splats, and audio. It is designed to integrate seamlessly into game engines, DCC software, and web applications for rapid asset creation.

VOID is a video inpainting model built on CogVideoX that removes objects and their physical interactions from videos, such as falling items or secondary effects. It uses a two-pass architecture with interaction-aware mask conditioning to maintain temporal consistency, leveraging VLM reasoning and SAM2 to generate precise masks for comprehensive scene editing.

Ghost Font is an experimental anti-AI communication tool that uses motion-based visuals to display messages readable by humans but difficult for AI models to interpret. By combining motion, noise, and decoy messages, it effectively thwarts current OCR and multimodal AI analysis, offering a creative approach to preserving human-readable communication in an era of advanced AI perception.