CTAC’s Lead Drupal Developer, Rob Morris, is the developer of the AWS Bedrock Module for Drupal which is an AI chatbot tool available on the Drupal Marketplace. In our recent webinar, he details some of the top AI Implementation best practices he learned along the way. Although our initial demo uses AWS, Claude and Drupal, these recommendations are relevant for any AI tool:

1. Data curation beats model size
2. Explicit system prompts give better guardrail efficacy
3. Observe & iterate – analyze logs for insights
4. Use role-based access to specific agents

1) Data curation beats model size

The AWS Bedrock tool will evaluate all data sources for the best answers. However clean, properly structured HTML / documents, proper tagging and metadata will always provide the best results. Even if you have some of the big, robust models out there to extrapolate the data and make correlations, they can only do their best to work with what they have. Nothing is going to beat having clean data. Well-structured, task-relevant content drives answer quality more than throwing a larger model at messy data. Garbage in = garbage out, just more confidently.

2) Explicit system prompts for better guardrail efficacy

Clear system instructions + structured policies narrow the model’s “degrees of freedom,” reducing unsafe or off-scope answers. Guardrail engines (Amazon Guardrails, Bedrock policy templates, etc.) evaluate every message. However, they weigh system instructions more heavily than user text. If a user message contradicts what is set in system instructions, it will not do it. Note that improvements here are slated for the next release of our AWS Bedrock module for Drupal, as we are always striving for better results.

3) Observe & iterate – analyze logs for insights

Your chatbot is a living system. The best performance gains come from continuous learning from real user behavior. Every unanswered question today becomes tomorrow’s content update to make things easier for the users. Depending on your organization’s user data policies, you could store anonymized transcripts for analysis to understand what users are actually interested in. For AWS users, our recommendation is to pipe logs to Cloudwatch Insights and review those regularly.

4) Role-based access to specific agents

Different users need different capabilities and content. In addition to implementing the principle of least privilege, role-segmented “agents” reduce risk, improve precision, and keep audits clean. For usage by account holders, specify which roles that can use particular chat interfaces that link to specific knowledge bases / agents. Our Bedrock chat module has another upcoming release related to this – instead of only a single configuration for the chat blocks, each individual block will have an override to allow its own customization by pointing to its own agent or knowledge base.

We hope these 4 tips will help you to get the most out of your new AI chatbot. For any further questions about implementing AI chatbots on government websites, reach out to info@ctacorp.com.