Natural Language to Gremlin: Empowering Graph Database Access

The article details an innovative framework for generating Gremlin queries from natural language using Amazon Bedrock models, specifically Amazon Nova Pro. This addresses the challenge of specialized query languages acting as a barrier for non-technical users, like business analysts, to efficiently extract insights from complex graph databases. The core benefit is democratizing access, enabling seamless interaction without requiring deep technical expertise in Gremlin or graph structures.

The solution operates in three stages. First, it extracts comprehensive graph knowledge, encompassing both structural elements like vertex/edge labels, properties, and one-hop neighbors, and domain-specific context. This domain knowledge is sourced from customer-provided constraints, exemplified by kscope.ai, and LLM-generated semantic descriptions for enhanced understanding. Second, the framework structures this graph information into a schema akin to text-to-SQL processing, facilitating the model’s comprehension. This involves question processing for entity recognition, context enhancement, and query planning, ensuring alignment with the database’s topology and business rules. Third, an LLM generates the initial Gremlin query, which is then executed. Crucially, an iterative error-handling mechanism refines failed queries using LLM-based feedback, ensuring the generated queries are accurate and executable.

Benefits include significantly improved accessibility and efficiency. Testing across 120 questions, with evaluation by Anthropic’s Claude 3.5 Sonnet on Amazon Bedrock, yielded an overall execution accuracy of 74.17%. Amazon Nova Pro demonstrated superior performance over a benchmark model, exhibiting lower query generation latency (median 1.8-4 seconds vs. 5-11 seconds) and reduced costs (below $0.005 vs. up to $0.025). Moreover, the generated queries maintained execution efficiency comparable to human-written ones. A key risk identified is the potential for LLM-based evaluation to misalign with human preferences, suggesting the necessity of involving subject matter experts to refine evaluation prompts. The framework’s success stems from its robust integration of graph and domain knowledge, RAG for query planning, and iterative error refinement.

(Source: https://aws.amazon.com/blogs/machine-learning/generate-gremlin-queries-using-amazon-bedrock-models/)