Medical KBQA: combining NER, graphs, and question answering

Why knowledge graphs still matter

Large language models are powerful, but structured knowledge still matters when the domain needs reliability.

Medical question answering is a good example. A user might ask about a symptom, disease, medicine, department, or treatment relation. A simple keyword search can miss the structure of the question. A free-form model can sound confident without a trustworthy source.

A knowledge graph gives the system a more explicit backbone.

Public evidence

This is now framed as a public case note rather than a public source release. The current evidence trail is the project write-up and the evidence page at /projects/medical-kbqa. If the source repository becomes public later, I will link it directly.

The question I care about is not whether the system can produce fluent text. The question is whether each stage is grounded enough to inspect and improve.

The pipeline

The Medical KBQA project uses a pipeline with several stages:

recognize medical entities in the question
map entities to graph nodes
infer the question intent
generate a Cypher query
retrieve structured answers from Neo4j
present the result through a web interface

The implementation uses BERT, BiLSTM-CRF style named entity recognition, Neo4j, Cypher, Flask, Streamlit, and medical NER components.

The goal is not to make the model sound smart. The goal is to connect a user's natural question to a structured answer path.

In plain terms: the system first tries to understand what medical thing the user is asking about, then it looks for the relevant relation in the graph, then it turns that structured result back into an answer.

Entity recognition is the front door

NER is one of the highest-leverage parts of the system.

If the system fails to identify the disease or symptom correctly, every later step becomes fragile. The graph can be well designed, and the query engine can work perfectly, but the answer will still be wrong if the question is grounded in the wrong entity.

That makes medical NER different from a generic tagging task. The model has to handle domain vocabulary, aliases, Chinese phrasing, and partial descriptions.

Graph queries make the answer inspectable

One advantage of a graph-based system is that the answer path can be inspected.

Instead of only returning generated text, the system can show that the answer came from a relation such as:

disease to symptom
disease to department
disease to medicine
medicine to indication
symptom to possible disease

That structure helps debugging. If an answer is wrong, the developer can inspect whether the problem came from entity extraction, intent classification, relation design, or graph data.

Where LLMs could help

This kind of system also suggests a balanced way to use LLMs.

The graph should handle structured retrieval. The model can help with question rewriting, intent disambiguation, answer phrasing, and follow-up suggestions. But the final factual answer should stay grounded in graph evidence whenever possible.

That division is important. In high-trust domains, generation should not replace retrieval. It should make retrieval easier to use.

What I would change next

The next version should expose more debugging information in the interface:

detected entities
predicted intent
generated Cypher query
matched graph relation
answer confidence or missing-data warning

Those details might be hidden by default for normal users, but they are extremely useful for development. A medical QA system should be easy to audit when it fails.

What I learned

Medical KBQA taught me that AI systems are not one model. They are pipelines.

Each stage has a different failure mode. Entity recognition can miss a term. Intent classification can choose the wrong relation. Graph data can be incomplete. The UI can hide uncertainty.

The engineering lesson is to make each stage visible enough to debug. A system that can explain its own path is easier to improve than a system that only returns an answer.