Talk
In aviation, search isn’t simple—people use abbreviations, slang, and technical terms that make exact matching tricky. We started with just Postgres, aiming for something that worked. Over time, we upgraded: semantic embeddings, reranking. We tackled filter complexity, slow index builds, and embedding updates and much more. Along the way, we learned a lot about making AI search fast, accurate, and actually usable for our users. It’s been a journey—full of turbulence, but worth the landing.
- Understanding how Postgres works a bit - Understanding about basic NLP - Some knowledge about AWS
In aviation, search is anything but straightforward. Reports are written by humans—pilots, cabin crew, engineers—each using their own mix of abbreviations, technical jargon, and everyday language. Standard keyword search often falls short. You might miss critical safety signals because a pilot wrote “navigation didn’t work” instead of “gps jamming,” or used a shorthand unknown to engineers on the ground. What we needed was semantic search—something that understands meaning, not just matches strings.
But we started simple with a plain Postgres setup. Our goal: build something that works. We began with pgvector and basic sentence embeddings to enable semantic search inside Postgres. It was scrappy, but it gave us just enough traction to prove the value of semantic search in this domain.
Then things took off. As complexity grew, so did the need for better retrieval and smarter ranking. We restructured the system: upgraded to better sentence embeddings, and most importantly, added reranking using cross-encoders. This turned our search results from “kinda relevant” to “spot on.” We moved to OpenVINO to make reranking faster on the CPU, especially important since we deploy on AWS Lambda.
But the technical challenges didn’t stop there. We experimented with different pgvector index types—IVFFlat vs HNSW—and discovered surprising trade-offs in index build times and performance, especially under constrained RDS instances. Embedding updates became their own problem, so we built a parallel processing system using SQS and a tool we call “Cockpit” to manage recomputation.
On top of that, search in our world isn't a single step. We layer semantic retrieval with full-text filtering, structured filters (e.g., airport, aircraft type), and real-time inputs. This creates a multi-layered AI search pipeline that needs to feel snappy and reliable to end-users.
In this talk, we’ll walk through how we made this work with minimal ML infrastructure, how we evolved from an MVP to a robust system, and what tools made the biggest difference—from tokenization strategies and inference optimizations to batching tricks and search composition patterns. You’ll also hear the gritty details: bottlenecks between tokenization and inference, indexing challenges, and lessons from building this in production for a safety-critical industry.
This talk is for folks who want to leverage Postgres for hybrid search as well. It’s for anyone who has ever duct-taped search with SQL and wondered how to take the next step. We’ll keep it real, share what we did, and reflect on what we’d do differently next time.
Partner & CTO @ DATANOMIQ / VP AI/ML Research & Engineering @ Beams
Dat is a seasoned technology and business leader with deep expertise in AI, machine learning, and digital transformation. As Partner & CTO at DATANOMIQ, he advises companies on AI strategy and implementation. Before that he worked for Axel Springer SE, idealo.de, Pivotal Labs and Accenture. His interests are diverse from traditional machine learning, deep learning, computer vision, AI in general to large language models. He has a lot of experiences from devising realistic data-driven use cases to the actual implementation into a real product; more than capable of distinguishing hype, buzzwords and wannabes from substance. He’s actively engaged with the global tech community, sharing insights on AI, tech leadership, and digital transformation with over 76k+ followers on LinkedIn. As a frequent keynote speaker, he has presented at conferences such as PyData, WeAreDevelopers, and many more, mentoring professionals in machine learning and leadership along the way.
Staff Software Engineer
Dennis is an engineering leader and product-focused technologist with deep expertise in building modern software systems, mobile platforms, and intuitive, user-centered products. As Staff Engineer at Beams, he helps shape AI-driven solutions for safety and risk management in aviation and beyond. Previously, he held senior engineering and leadership roles at Pivotal Labs and SoundCloud—helping scale teams, launch cross-functional product initiatives, and drive iterative development practices for large clients such as Volkswagen and across sectors like banking, insurance, and multimedia. A passionate builder, Dennis has co-founded multiple startups and continues to run several side projects, among which is an “App of the Day” winner in the US. He works across languages and stacks, focusing on long-term maintainability, thoughtful architecture, and delivering real-world impact.