AivexaNewsSearch
AI news for builders and product teamsChecked every hour

How ShipStation Global built an AI IT help desk on Amazon Bedrock AgentCore

Collected Oct 13, 2026

ShipStation Global's engineering team built an AI-powered IT help desk agent called Deckhand that runs inside Slack on Amazon Bedrock AgentCore. The company, which describes itself as a shipping and logistics platform for small and mid-sized shippers, published details of the architecture, the data pipeline behind it and its early operating results. The post is co-written with Kenneth Salmon, Edward Boswell, Jack Armstrong, Robert Banh and Nick Ciubotariu from ShipStation Global.

The problem will be familiar to most IT organizations. In the 12 months before launch, employees posted more than 1,500 threads in a busy help desk Slack channel, and volume kept growing. Answers already existed in Confluence how-to articles, intranet pages and PDFs, IT-maintained guides, and older threads, but finding them was the hard part. ShipStation has employees in the US, UK, Spain, Poland, Australia and the Philippines, and more than a fifth of questions arrived outside US business hours. Chris Collins, Senior Director IT at ShipStation Global, said the team wanted getting help to be as simple as asking a question in Slack rather than deciphering a ticket form, and described tickets as an indication of operational friction rather than productivity.

Deckhand joins Slack channels like a teammate. It reads conversations, pulls context from Slack profiles and answers in place. The stack has four main parts. A Slack service built with FastAPI and Slack Bolt runs on AWS App Runner, receives Slack events, decides whether the bot should respond and assembles a user-context payload from the requester's profile: name, email, time zone and manager. An agent service is deployed as a container on Amazon Bedrock AgentCore runtime and manages the multi-turn tool loop, using Anthropic's Claude in Amazon Bedrock. Tools and knowledge run through a single AWS Lambda function exposed as MCP tools via AgentCore Gateway; it classifies requests, creates and updates Jira Service Management tickets and analyzes images. An Amazon Bedrock knowledge base indexes content from Confluence, the company intranet, Slack and marketing pages, stored in Amazon S3. Operations and feedback use DynamoDB for ratings, AWS X-Ray with OpenTelemetry for tracing, and CloudWatch for logs.

The agent itself is deliberately small. It is written with the open source Strands Agents SDK, where an agent is a model, a set of tools and a system prompt, and the SDK manages the reasoning loop. Model configuration uses a maximum of 2,048 tokens and a temperature of 0.3, with the model ID supplied through an environment variable so models can be swapped without code changes. Retrieval against the knowledge base is a native Strands tool imported once. Everything else, including classification, ticket management, user resolution and image analysis, goes through the Gateway-registered Lambda, which routes each call by tool name. That split means tool logic can change independently and the agent can pick up new tools without a rebuild.

The most interesting part of the write-up is how ShipStation turned its own Slack history into a knowledge base. Years of resolved support threads are an organic, human-validated corpus, but a raw export mixes banter, automated alerts and threads that were never actually resolved. Indexing a bad fix means the system later serves an incorrect answer with the same confidence as a correct one, so extraction has to be strict. The pipeline exports the channel history through the Slack API with full thread replies, then uses DSPy's Recursive Language Model running Claude on Amazon Bedrock. Rather than a single prompt, the RLM treats the export as a memory space inside a sandboxed Python REPL and works over it iteratively. Threads are classified as resolved, unresolved or escalated, with announcements and non-questions filtered out. A thread qualifies as resolved only when the full, workable fix appears in the thread itself. For each qualifying thread, recursive sub-LLM calls extract structured fields validated against a Pydantic model: issue summary, original question, resolution, whether the fix is self-service or needs a ticket, the steps, the application and system involved, error messages and tags. Validation errors are caught in the interpreter and retried. Output is a markdown document organized by issue, uploaded to S3 for indexing.

Deciding when not to answer gets as much attention as answering. On every message the agent first judges whether it is an announcement, a social message or a conversation between people; if so it returns SKIP and the Slack service suppresses the reply. In threads it replies only to the original poster. If the last reply came from another human helper it stays silent, and any reply that opens by naming a person is treated as not for the bot. This is a sensible design instinct: a bot that talks over people erodes trust faster than one that misses an occasional question.

Each answer carries feedback buttons. A thumbs-up is recorded directly; a thumbs-down asks briefly what went wrong. Ratings land in DynamoDB alongside the original question and the response. Negative feedback feeds back into the system prompt, agent behavior and the knowledge base, which is the loop that appears to explain the improvement in scores.

Security controls are enforced in code and infrastructure rather than left to the prompt. Ticket creation needs explicit confirmation through a Create Ticket button. Ticket updates accept only keys matching the helpdesk project pattern and reject empty comments. Classification runs as a separate Amazon Bedrock Converse API call returning structured ticket fields, validated with Pydantic before anything reaches Jira. The image tool downloads only from Slack's file host, authenticates with the bot token, caps each request at five images of up to 5 MB and sends them to a vision model via the Converse API. Slack tokens, the signing secret and Jira credentials live in AWS Secrets Manager, and each service and tool runs under its own scoped IAM role. The prompt also forbids requesting passwords or MFA codes, writing credentials into ticket comments and exposing profile metadata used for ticket fields.

Results are concrete. Deckhand launched in April 2026, started at roughly 30 messages a day and reached as many as 100 by early summer. In its first three months it handled 1,935 messages, started 1,221 agent conversations and opened 409 Jira tickets, replying to 97.5 percent of messages. A third of questions arrived after hours, and Deckhand answered 94 percent of those. Positive ratings rose from 82 percent in April to 96 percent in June after the team revised the system prompt and knowledge base.

The ticket deflection math is the clearest argument for the approach. Of 1,221 agent conversations, only 409 became tickets; the other 812 were handled in chat. At roughly 30 minutes to file, work and close a ticket, that is about 406 hours over three months, or around 1,600 hours a year.

The pattern is available to copy. Prerequisites include an AWS account with Bedrock, AgentCore runtime and AgentCore Gateway access; a Slack app with event subscriptions, a signing secret and a scoped bot token; a container registry such as Amazon ECR; an S3 bucket and Bedrock knowledge base; and scoped IAM roles plus Secrets Manager entries. The post points readers to the AgentCore, Strands Agents SDK and Bedrock Knowledge Bases documentation.

Why it matters: this is a full worked example of an internal agent that a platform team could replicate, and the transferable pieces are the managed runtime, the Gateway-backed tools, and above all the disciplined mining of a company's own chat history into a retrieval corpus. For developers building similar agents, the takeaway is that answer quality depends less on the model than on curation, schema validation and a clear skip policy, while the deflection numbers suggest the payoff is measured in reclaimed IT hours rather than ticket volume. One caveat, as inference rather than a reported fact: these results come from a single help desk in one company, so teams with messier history or thinner documentation should expect lower coverage at first.

Read at AWS Machine Learning Blog

Based on reporting from the original publisher. Visit the source for full context and later updates.

Publisher excerpt

ShipStation Global's engineering team built Deckhand, an AI IT help desk agent for Slack on Amazon Bedrock AgentCore. It answers routine IT questions from a curated knowledge base, reads error-message screenshots, and files Jira tickets. In its first three months it resolved 97.5% of channel queries with a 96% positive feedback rating.