Salif Sawadogo

• RAG-specific scoring: answer correctness and similarity, answer relevancy, faithfulness (the hallucination rate), contextual precision and contextual recall. Retrieval and generation scored separately, because they fail for different reasons and the fix is never the same.
• Deterministic checks alongside the judge: required key facts and must-contain assertions. Some things should not be graded on a 10-point scale. They are either present or the answer is wrong.
• LLM-as-judge done carefully: pairwise comparison with 3-vote majority, randomized presentation order to kill position bias, calibrated rubrics for faithfulness, helpfulness and safety.
• Golden datasets built with the business, not by the model. Synthetic data catches technical instability at scale but cannot capture the nuance and policy references a domain expert carries. Every case needs three parts: the question phrased the way a user would actually ask it, the answer an expert would write, and the source document that proves it. Plus negative tests: the questions the system must refuse.
• SLI/SLO definition and enforcement, with release gates that block a promotion when the numbers miss.
• Tracing and monitoring: Langfuse per-request traces, token-level streaming, tool-call visualization for multi-agent flows.
• Drift detection: Kolmogorov-Smirnov and Population Stability Index, with configurable alerting.
• Multilingual evaluation where exact match and F1 break down: BLEU, METEOR, TER, chrF.
• Statistical rigor most eval work skips: power analysis before the test, bootstrap confidence intervals, McNemar for paired binary outcomes, alpha-spending to stop teams peeking their way to a false positive.

Typical engagement: you have a RAG or agent system in production and no idea if last week's prompt change made it better or worse. I give you a scored eval suite, a dashboard, and a release gate. Two to four weeks.

• Hybrid retrieval: BM25 plus dense embeddings, with reranking. Vector stores including Azure AI Search, LanceDB and pgvector.
• Document ingestion at scale: crawling, Docling parsing, chunking strategies, metadata and breadcrumb-based section retrieval.
• Agent orchestration with LangGraph, pydantic-ai, LangChain, and MCP tool integration.
• Agent harnesses and tool calling. The framework is the easy part. The harness is what decides whether an agent is usable in production: typed tool contracts so a malformed call fails at the boundary instead of three steps later, tool surfaces scoped per use case rather than handing every agent everything, explicit termination and step budgets so a loop cannot run up a bill, retry and fallback behaviour when a tool errors or returns nothing, and per-turn logging that makes a bad trajectory readable after the fact. Model Context Protocol where tools need to be shared across systems.
• Multi-agent routing: structural locks for deterministic paths, LLM routers for semantic ones, with fallbacks when a branch returns too little context.
• LLM gateway patterns: one routing layer so applications never hardcode a provider SDK, which makes model swaps and cost control a configuration change.

Typical engagement: your RAG answers are plausible but wrong, or your agent loops. I audit retrieval quality separately from generation quality, fix the layer that is actually failing, and leave you the eval suite that proves it.

• CI/CD with automated validation gates: code quality, model validation against golden datasets, infrastructure compatibility, dependency scanning.
• MLflow Model Registry with staging to production promotion, approval gates, lineage tracking, and rollback to any prior version.
• Blue-green and canary deployment, shadow traffic, automatic rollback on SLO violation.
• Kubernetes (AWS EKS), Docker, Terraform, Azure Container Apps, serverless GPU inference.
• Offline-first architecture: IndexedDB and SQLite cache-first reads, mutation queues, auto-replay on reconnect, AES-GCM encryption at rest.
• On-device inference: quantized speech and language models running entirely on mid-range Android hardware, benchmarked on the phones users actually own rather than a flagship. Zero network, zero per-request cost, nothing leaving the device.
• Privacy by design: GDPR-compliant pipelines, PHI encryption, multi-tenant isolation enforced at the database layer, consent modelling with scope and expiry.

Typical engagement: you need AI in an environment with real constraints (health data, air-gapped, intermittent connectivity, a regulator). I design the architecture and the compliance story together, because retrofitting the second onto the first never works.

Where a model output drives a decision that costs something to get wrong, I make the uncertainty explicit and calibrated.

• Conformal prediction, which wraps a model you have already trained and returns intervals or prediction sets with a finite-sample coverage guarantee, distribution-free and model-agnostic. For regression, locally adaptive variants and conformalized quantile regression, so the interval widens where the model is genuinely unsure instead of staying a constant band. For classification, prediction sets rather than a single forced label.
• Judging the intervals themselves, not just producing them: marginal coverage against the target, average interval width, adaptability across regimes, and the Winkler score when you need one number that penalizes both an interval too wide to be useful and one too narrow to be honest.
• Knowing where the guarantee stops. Coverage is marginal, not conditional: 90 percent overall can hide 60 percent on the subgroup you care about, which is why I check coverage per segment. And the framework assumes exchangeability, so it degrades under distribution shift and needs weighting or an adaptive scheme when the data drifts.
• Classical uncertainty where it fits better: bootstrap confidence intervals, constrained estimation to tighten standard errors when the sign of an effect is known a priori, and power analysis before an experiment rather than after.
• Cost-sensitive decision thresholds. Tuning the operating point against the actual business metric and asymmetric error costs, rather than defaulting to 0.5 and accepting whatever the confusion matrix gives you.
• Calibration, so that a predicted probability of 0.8 means the event happens 80 percent of the time, which is the precondition for any of the above being worth anything.

Typical engagement: your model is accurate enough on average and still makes expensive mistakes, because nothing downstream knows which predictions to distrust. I add the uncertainty layer and tune the decision rule against your actual cost of error.

• Geomarketing and trade area analysis: catchment area modelling, spatial indices, point-of-interest and footfall enrichment, geodesic distance over proper projections. Answering which outlets are comparable, and why.
• Entity resolution and record linkage: fuzzy matching internal customer master data against external sources when there is no shared key, with human-labelled ground truth rather than an unvalidated similarity threshold.
• Customer segmentation: K-means and hierarchical clustering on behavioural and spatial features, with segments profiled in language the commercial team uses, plus a classifier to assign new accounts.
• Demand modelling and elasticities: log-log fixed-effects specifications that yield price and cross-price elasticities directly, with cannibalization and halo effects modelled explicitly through a substitution matrix.
• Assortment and planogram optimization: greedy or constrained range construction that accounts for substitution as the shelf fills, against a volume or margin objective. SKU rationalization on the same machinery.
• New product forecasting: analog or like-item modelling for SKUs with no history, using attribute-space similarity.
• RFM, customer lifetime value, and churn: including survival analysis when the business needs to know when, not just whether.
• Demand forecasting: SARIMAX where the seasonal structure is the signal, gradient boosting where it is not.

Typical engagement: you have transaction data and a commercial question (what should we stock, what will it sell, which customers are worth defending) and no analytical layer between the two. I build it, and I make it rerunnable next quarter rather than a one-off deck.

• Backbone: LiteLLM gateway routing every LLM call in the organization, Langfuse for traces, MLflow for experiment tracking. Model pricing is configuration, not code, so cost control and provider swaps do not require a deploy.
• Evaluation as a shared service. This is the part I am proudest of. Rather than each bot team improvising its own scoring, there is one FastAPI service every assistant calls: question and expected-answer datasets scored on answer correctness, relevancy, faithfulness, contextual precision and contextual recall, alongside deterministic key-fact and must-contain checks. Judge calls route through the same gateway as everything else. Jobs run async, results land in MLflow. Every bot on the platform now ships with the same standard pair of deliverables: an evaluation dataset built with the business, then a published campaign score.
• Ingestion: crawler over the bank's document management system, Docling parsing, blob storage plus Azure AI Search indexing. Handles project appraisal reports, country strategy papers, completion reports and thematic evaluations across member countries, in French and English.
• Assistants: six independent RAG agents (operations quality, SharePoint knowledge, economics, non-sovereign operations, evaluation results, sustainable development goals), isolated from each other so one team's release cannot break another's, but all sitting on one shared core package.
• One agent framework, deliberately. The bots run on pydantic-ai, wired to the gateway through a shared model adapter, with typed tool contracts and a common turn logger. Standardizing this was a decision, not an accident: before it, every bot had its own orchestration, its own history handling and its own way of failing. One assistant remains on LangGraph because its multi-step graph genuinely needs explicit state transitions, and forcing it to converge would have cost more than it returned. Picking one default and documenting the exception is the part most platform work gets wrong.
• Document generation: pydantic-ai agents that draft concept notes and score project readiness across four dimensions, using a router plus specialist plus synthesis architecture, with structural locks for the routes that must be deterministic and an LLM router for the rest.

A virtual sensor for aircraft brake wear, and a patent

Checking how worn an aircraft brake heat sink is normally means sending someone to inspect it on the ground. We replaced the inspection with an estimate: read the temperature sensors during the parking phase after landing, measure how long the brake takes to cool, and infer wear from that.

The difficulty is that cooling time is not a clean signal. Ambient temperature, wind, and the conditions on that particular apron all move it, so the same brake in the same condition looks different in Dubai and in Oslo. The method therefore models the environment explicitly, using XGBoost regression to predict how those conditions shift the expected cooling time, and reads wear from what the environment cannot explain. The result is per-aircraft, per-brake monitoring across an entire fleet, continuously, with nobody going out to look.

Filed by Safran with three named inventors, including me. French priority August 2024, PCT filing July 2025, published as WO2026032835A1 in February 2026. Unlike everything else here from Safran, this one you can read in full: a published patent is a public disclosure by definition, so the method, the modelling choice and the reasoning are all in the document.

Generative AI in production

• Multi-agent chatbot built from scratch on an internal LLM-as-a-service layer, with tool use and MCP integration.
• Evaluation framework with enforced service level objectives across factual accuracy, judged response quality, toxicity, PII exposure, latency and availability, wired to release gates that blocked a promotion whenever a target was missed.
• Multi-layered observability: Langfuse traces, MLflow and Prometheus metrics, Grafana dashboards over the Kubernetes layer, with cost monitoring and budget alerting.
• Release discipline built around weekly deployments, short lead time from commit to production, fast recovery, and a low change-failure rate, all tracked rather than asserted.
• AWS data science platform on EKS, built for the team's scalability and security requirements.

• Split into an assessment service (business logic, consultations, criteria, scores, exports) and an intelligence service (extraction, analysis, scoring), communicating by webhook so a long LLM job never blocks a request.
• Language normalization before scoring: text is extracted, language detected, then translated to a single working language. Without this step a French CV and an English CV are scored on different footings, which is the kind of bias nobody notices until an unsuccessful candidate asks why.
• Retrieval-backed scoring against the terms of reference, not freeform prompting, so each score traces to specific evidence in the document.
• Server-sent events for progress, Excel export, ranked shortlist output.

This one generalizes well. Any workflow of the form "score many documents against one rubric, and be able to justify each score" is the same system.

The platform work I do at Safran and the African Development Bank sits behind NDAs. This is the same architecture, open, so you can read it before hiring me: github.com/sawallesalfo/DS-backbone

One docker compose up gives a team the full loop: JupyterLab for exploration, MLflow for experiment tracking and the model registry, PostgreSQL as MLflow's backing store, MinIO as S3-compatible artifact and dataset storage, and Nginx routing it all behind clean hostnames instead of a spread of localhost ports.

The point is what it removes. Most teams assemble these five pieces by hand, differently on every laptop, then spend the first month of a project debugging why an experiment that ran locally will not reproduce. Configuration is environment variables, storage is S3-compatible from day one, so moving from this sandbox to real cloud storage is a URL change and not a rewrite.

If you are hiring someone to set up your data or ML environment, this repo is the deliverable, and you can inspect it before we talk.

The most complete piece of work I can show end to end, because the report is public and so is the core algorithm.

ECCBC knew its own points of sale but knew almost nothing about the neighbourhoods they sat in. Every HORECA outlet in the channel received the same assortment, because there was no basis for giving them different ones. Pilot zones Marrakech and Tizi Ouzou. The chain runs: enrich the customer base with external spatial data, segment on it, model demand per product within each segment, then use those models to decide what goes in each fridge.

• Entity matching under noise. Internal records had no shared key with any external source. I built a composite similarity score over seven string metrics (Damerau-Levenshtein, Jaro-Winkler, n-gram overlap and others), weighted 75 percent on name, 15 percent on address, 10 percent on geodesic distance, searching a 2 km radius. Matched a 5,457-entry TripAdvisor corpus against internal records. Validated it the honest way: a stratified 400-record sample checked by hand to produce labelled ground truth, then treated as a supervised problem.
• Spatial enrichment. Trade area analysis, spatial indices, geodesic distance over proper reference ellipsoids and projections, plus OpenStreetMap and Flickr signals for footfall and points of interest.
• Segmentation. K-means with K-means++ initialization, benchmarked against hierarchical clustering, on standardized features. Eight interpretable segments for cafés and restaurants, three for hotels, each with a business-readable profile (Premium and Entertainment, Semi-Rural, Low Affluence and Mobility, and so on). Points of sale outside the clustering sample were assigned by Random Forest.
• Demand models, one per SKU per segment. Weekly data, 2019 to 2021, log-transformed volumes with a fixed effect for baseline throughput, a cannibalization matrix across brands and flavours, and external regressors for weather and events. Variable selection was automated: drop near-zero variance, drop correlations above 0.8 unless they came from the cannibalization matrix, drop VIF above 10. The socio-economic variables fell out on their own, which was the right result: the clustering had already absorbed that variance, and a model that kept them would have been double-counting.
• Constrained and unconstrained versions of each model. The constrained variant imposes sign constraints where the business meaning is unambiguous: cannibalization effects cannot be positive, fridge doors and merchandising quality cannot hurt volume. This narrows the standard errors and tightens the confidence intervals, which matters when the output drives a purchasing decision rather than a slide.
• Validated properly, at scale. Shapiro-Wilk for normality, White for homoscedasticity, Durbin-Watson for autocorrelation, all above 0.05 across every modelled SKU. MAE and MAPE computed after back-transforming out of log space, because an error term in logs tells a commercial team nothing. Worst test MAPE across all regressions: 14 percent. Since there were far too many SKUs to inspect by hand, the pipeline auto-generated a quality report per model: fit statistics, coefficient distributions, constraint bounds, and hypothesis test results.
• Cold start on new products. A SKU launched this year has no history, so it cannot be modelled. Rather than dropping it or guessing, I flagged it as limited-coverage and estimated it by proxy: find the three nearest SKUs by cosine distance over product attributes, then take a similarity-weighted average of their predictions, calibrated against their observed sales. New products are precisely the ones a category manager most wants an answer on.
• The assortment optimizer. Greedy rank construction from an empty fictional fridge. Rank 1 is the best-performing SKU with cannibalization switched off, since nothing else is on the shelf yet. Rank 2 re-estimates cannibalization against that first product and picks the best addition. Rank j repeats, always conditioning on what is already in the fridge. The output is an ordered assortment per segment, benchmarked against the ranks actually observed in sales. The objective function is a parameter: maximize volume, or maximize margin.

Insurance pricing is one of the few places where a model has to satisfy a regulator as well as a business. I migrated the pricing stack from traditional GLMs to machine learning without losing that: separate frequency and severity risk models meeting financial regulatory requirements, a big data migration to Azure with CI/CD integration, GDPR-compliant handling throughout, and continuous performance monitoring after release. Processing time fell 40 percent.

The interesting constraint is explainability. An actuary can defend a GLM coefficient to a supervisor. Replacing it with a gradient-boosted model means you now owe an account of why the price moved, which shapes what you are allowed to build.

Before joining, I won first place in AXA Direct's internal data science competition. Code is public, written in R.

Replaced keyword matching with BERT-based semantic embeddings, on top of NLP-based deduplication and normalization of a messy product catalogue.

Then measured it properly. Not a simultaneous A/B split: a gradual rollout over four weeks, which means randomization was imperfect and the naive before-and-after number could not be trusted. So the lift was estimated with causal inference. Causal Forest for heterogeneous treatment effects across user segments, Double Machine Learning to control for confounders, propensity score matching to correct the selection bias the rollout design introduced.

+29 percent user satisfaction, significant at p < 0.01, and defensible because the seasonal and behavioural drift was modelled out rather than assumed away. The number I would have reported without that layer was larger and partly fictional.

Built around seven questions a retailer actually asks: who are my best customers, who is about to leave, who could be worth more than they are, who is already gone, where does retention effort pay off, who is loyal, and who will respond to this campaign. Public repository.

• RFM segmentation as the foundation, because recency, frequency and monetary value are interpretable by the commercial team, and a segmentation nobody understands never gets used.
• Customer lifetime value modelling on top of it.
• Churn and attrition treated as survival analysis rather than a binary classifier, so the output is when a customer is likely to leave, not just whether.
• Sales forecasting at daily and weekly granularity, SARIMAX for the seasonal structure, XGBoost and LightGBM where the relationships are not linear.
• Supply chain movement monitoring on the same feature base.

It is organized as a pipeline (inputs, intermediates, then one folder per analysis) rather than a pile of notebooks, which is the difference between an analysis someone can rerun next quarter and one they cannot.

A non-profit in Ouagadougou building speech and translation models for a language with almost no digital corpus.

• Translation: fine-tuned NLLB (600M and 1.3B) and Mistral-7B for French to Mooré.
• Speech recognition: fine-tuned Whisper Large, three iterations.
• Speech synthesis: SparkTTS (Qwen2 backbone, LoRA), XTTSv2, VITS, ParlerTTS.
• Public leaderboards for translation (BLEU, METEOR, chrF), ASR (WER, CER) and TTS, against a 1,483-sample benchmark across five domains.
• The interesting part: a synthetic data flywheel. Real speech trains the TTS model, the TTS model generates synthetic audio that augments the ASR corpus, the improved ASR transcribes more real audio, which yields more text pairs for the next TTS round. That is how you get past a 1,000-pair starting corpus without funding a large recording campaign.

Everything is public: huggingface.co/burkimbia and github.com/BurkimbIA

Multi-tenant platform connecting patients, clinicians and institutions, built for regions where specialist care means travelling and connectivity is not guaranteed.

• Offline-first by design: patient and clinician apps work with no network. Cache-first reads, queued mutations, automatic replay on reconnect, AES-GCM encryption for health data at rest in the browser.
• Two-level GDPR consent: storage consent, then per-clinician access consent with explicit scope and expiry.
• Tenant isolation enforced at the database level, with users able to belong to multiple institutions.
• Medical transcription service: streaming ASR to structured clinical notes.
• Conversational health agent with hybrid vector and BM25 retrieval.

Running the whole clinical assistant on the phone. Cloud transcription is the easy answer and the wrong one here: it needs a network the clinic may not have, and it sends patient speech to a third party. So I built the offline path and measured it. Whisper via whisper.rn for speech, a small language model via cactus-react-native for the clinical note draft, benchmarked on the phones clinicians actually carry (Tecno, Infinix, Samsung A-series) rather than a flagship: Whisper tiny against small, Gemma 270M against Qwen3 0.6B, on latency and quality. Models download once, then nothing leaves the device.

That work became a step-by-step guide to deploying an LLM on mobile, written for backend and data engineers who have never opened Android Studio.

Retail shops and wholesalers in Ouagadougou. The product is not a cash register, it is a decision assistant: margin, trend, what actually earns, what is sitting dead on the shelf. Analytics is the product.

Offline-first out of necessity: local SQLite is the source of truth, a sync engine watches connectivity and pushes queued sales in batches when the network returns. Multi-tenancy is enforced with Postgres row-level security rather than application code, so a query bug cannot leak one merchant's sales into another's.

The analytics layer is where the real work is, and three rules separate it from a dashboard:

• Margin is only computed on lines where the purchase price was actually known, and every figure carries its coverage. A margin presented as complete while silently ignoring half the sales makes someone decide on a false number.
• An unknown margin displays "not calculated", never zero. Zero margin and unknown margin call for opposite actions.
• Cost is snapshotted at the moment of sale. Wholesalers change their prices without warning. Reading the current purchase price at calculation time would silently rewrite last month's margins.

While building it I audited a neighbouring prototype's inventory formulas and found the days-inventory-outstanding calculation was dividing a quantity by an amount in currency, using closing stock where its own comment specified average, and multiplying by the period after the division had already produced one. On a real case (350 F purchase price, 26 units average stock, 8 sold in 14 days) the prototype returned 0.16 days against a true 45.5. The error scaled with purchase price, so it was not even a constant offset. The dashboard had been showing plausible, meaningless numbers.

Testing is deliberately unfashionable here: 164 mobile tests run against a real SQLite engine, and 9 backend tests against a real Postgres. I refused jest-expo because its automatic mocks execute no SQL, so the suite would pass without proving anything.

Monthly series from 2004. Stationarity testing (Augmented Dickey-Fuller), Johansen cointegration, VAR and VECM specification with information-criterion lag selection, Granger causality, impulse response functions, and forecast error variance decomposition at a 12-month horizon. Full residual diagnostics: Jarque-Bera p = 0.38, White p = 0.45, Portmanteau p = 0.91. Included a COVID-19 shock analysis on the price level. Slides public.

I list this because time series and causal identification keep coming back. Forecasting, anomaly detection, and any question of the form "did our change cause this" are the same toolkit.