Modernize application architectures with IBM Bob

Application modernization is rarely quick or straightforward. Legacy systems are often deeply embedded in critical business operations, sparsely documented, and built on assumptions that no longer hold. Reengineering them without disrupting the business they support is a significant challenge.

IBM Bob can accelerate architecture modernization by helping you analyze trade-offs, compare technologies, and generate structured proposals — but it works best as a design companion, not a replacement for critical architectural thinking.

To better understand the complexity of application architecture, we will review a fictional application, Dates and Beans, a commodity classification application that categorizes agricultural products for international trade compliance. The Dates and Beans application generates standardized classification codes for items in the catalog, a critical function that enables clients to meet regulatory requirements, streamline customs processing, and maintain accurate inventory management across global supply chains.

After dissecting the existing architecture, we present the skeleton of an expected solution and then show how to use IBM Bob to reengineer it and make it business-ready. We compare our expected solution to the proposals from IBM Bob using a simple, fast prompt in Ask mode and then a more complex, detailed prompt in Plan mode.

Introducing the sample application architecture

What started as a manageable classification system (the Dates and Beans application) for a few dozen product types has evolved into a complex operation handling 10&sup8; distinct product categories. Each product receives a hierarchical 8 digit code that captures increasingly specific attributes.

The product classification system generates hierarchical codes for product types, primarily focused on date-related and bean-related commodities. The system descriptions are mapped to standardized classification codes following this structure:

• Beans: 01000000

  • Red Beans: 01010000

    • Big Red Beans: 01010100

      • Spanish Big Red Beans: 01010101
      • English Big Red Beans: 01010102

    • Small Red Beans: 01010200

  • White Beans: 01020000

Given the product catalog of 10&sup8; distinct items and the hierarchical nature of the coding system, the system must handle significant computational complexity (10&sup8; potential code combinations).

Challenges in the current architecture

The current architecture presents the following critical challenges:

• Legacy processing requirements: Each processing step requires manual intervention and uses a fixed schema that is not required for the application to run smoothly.
• Data preprocessing overhead: Records must be manually preprocessed within our cloud infrastructure before classification can begin.
• Error susceptibility: The manual nature of the workflow introduces significant risk of human error throughout the process.
• Excessive operational costs: Unsustainable expenses related to cloud storage, large language model (LLM) API consumption, and overall processing overhead for this classification task.
• Poor client experience: The requirement to upload complete datasets to our cloud infrastructure, multi-day turnaround times for code generation, and overall process design that does not meet the operational needs.

As Dates and Beans' business grew, so did the weaknesses in the architecture. What once worked for smaller datasets now struggles under the weight of modern demands: manual preprocessing bottlenecks, escalating cloud costs, multi-day processing times, and increasingly frustrated clients who expect near-instant results in today's digital economy.

Analyzing the current architecture in detail

Before we can propose solutions, we need to understand exactly what's happening under the hood. The following diagram illustrates the current system architecture, which is a classic cloud-based processing pipeline that, while functional, embodies many of the antipatterns that led to the challenges.

This architecture represents what we call a "cloud-first, optimization-later" approach, a common pattern where initial implementation prioritizes getting something working over long-term efficiency. The result is a system that technically functions but fails to meet business requirements for cost, speed, and user experience.

Target architecture for a modernized application

To modernize the architecture of this sample application and address the challenges, the expected architecture would be similar to the following diagram.

In the original architecture, there was not a front end interface, so we will assume that there is no need for it. Also, we will omit several data preprocessing steps since they are not required for the application to run and assume that the application user will send exactly what is required.

The modernized architecture will need to be able to:

• Run the main workflow to infer codes in parallel and handle multiple parallel requests.
• Handle queues, different databases, and multiple algorithms by using a workflow orchestrator like Conductor OSS.
• Decouple the client from the workers, and in order to prevent downstream overload failures, use Conductor OSS PostgreSQL queue persistence through the orchestrator, backed by PostgreSQL. PostgreSQL can store the result of the workflows together with the reference promise, and it can also work as the vector database.
• Have a queue manager like Kafka using Kafka security features and allowing external users to subscribe to Kafka topics using Python clients. The payload will be the description only, as this is the base for infering the codes.
• Have each worker decide the complexity of the query. This selector will determine if the input should be parsed using a deterministic or probabilistic algorithm. If the query is simple enough, it can use something like a exact match lookup or a fuzzy match lookup. In the case where the descriptions are a bit more complex, it can use k-nearest neighbors (KNN). Lastly, if the information is too complex, it can use more refined methods like LLMs. In case a probabilistic method fails after an N number of retries, the workflow will default to the most advanced deterministic method.
• Store the taxonomy of all the existing codes, which in this case it can be the PostgreSQL with pgvector.
• Store all the results of the workflow in PostgreSQL, which can be queried later by a backend algorithm or by the user directly.

Improving the architecture using Ask mode

According to the IBM Bob Ask mode documentation, you use Ask mode when you need explanations, documentation, or answers to technical questions. It is best for understanding concepts, analyzing existing code, getting recommendations, or learning about technologies without making changes. Using this mode will allow us to get a quick overview of what a redesigned architecture could look like.

The context you feed into Bob and the way you ask for information matter a lot. We tried to craft a prompt that would contain as much information as possible about the current issues that the client was facing:

Hi, I need help redesigning a system. Here's an email from our client describing the problems, and a diagram of the current architecture.

We have a product classification system that turns product descriptions (like "Spanish Big Red Beans") into 8-digit hierarchical codes.

The current setup is: user manually cleans records, uploads everything to cloud storage, a producer creates tasks on a message queue, workers pick them up, call an LLM for each record to get the code, and save results back to storage. Everything runs on a VM in the cloud.

Problems:
- too many manual steps
- way too expensive (LLM API costs, storage, the VM)
- clients hate uploading their whole dataset to us
- results take days due to communications mismatches between client and us

Can you propose a better architecture? Something automated, cheaper, and faster. Include a diagram and explain why your version is better.

Mail:
What started as a manageable classification system (the Dates and Beans application) for a few dozen product types has evolved into a complex operation handling 10^8 distinct product categories. Each product receives a hierarchical 8 digit code that captures increasingly specific attributes.

The product classification system generates hierarchical codes for approximately 10^8 product types, primarily focused on date-related and bean-related commodities. The system descriptions are mapped to standardized classification codes following this structure:

- Beans: 01000000

  - Red Beans: 01010000

    - Big Red Beans: 01010100

      - Spanish Big Red Beans: 01010101
      - English Big Red Beans: 01010102  

    - Small Red Beans: 01010200

  - White Beans: 01020000

Given the product catalog of 10^8 distinct items and the hierarchical nature of the coding system, the system must handle significant computational complexity (10^8 potential code combinations).

Current Architecture:
graph LR
    A@{ shape: circle, label: "User" }
    B@{ shape: rect, label: "Producer" }
    C@{ shape: bow-rect, label: "Message Queue" }
    D@{ shape: procs, label: "Workers"}
    E@{ shape: hex, label: "LLM" }
    F@{ shape: lin-cyl, label: "Cloud Storage" }
    H@{ shape: rect, label: "Preprocessor" }

A --  1. Clean Records --> H
A --  4. Convert Records to Tasks  --> B

subgraph Cloud Provider
    direction BT
    C
    E
    F
    B --  5. Send Tasks  --> C
    C --  6. Send Task  --> D
    D --  7. Get Task Data  ----> F
    D --  8. Get Codes  --> E
    D --  9. Store Result  --> F
    H --  2. Get Records  --> F
    H --  3. Save Cleaned Records  --> F
    subgraph Virtual Machine
        direction RL
        B
        D
        H
    end
end

Our answer may differ from the answer you will get. In our case, Bob gave an answer consisting of several parts. Firstly, it described the pitfalls of the current design, with some valuable insights. After this, Bob provided a diagram of the new proposed architecture.

After this, Bob explained how it solved each one of the issues that the original architecture had.

Finally, it explained how the matching works, along with a comparison of both architectures and a semi-bill of materials (BOM) of the items needed to build the proposed solution.

Bob's proposal presents several commendable improvements alongside some points that warrant further consideration.

Points of agreement with the Ask mode proposal

The proposed architecture contains several design decisions we agree with and would carry forward into any revised approach:

• Elimination of cloud uploads: The removal of the manual upload step is a sound decision, as it introduced unnecessary friction and data retention risk with no architectural benefit.

• Serverless compute: Moving to a serverless execution model analogous to platforms such as IBM Code Engine is well aligned with our cost reduction objectives. By scaling to zero during periods of inactivity, we avoid the continuous billing associated with the current virtual machine setup.

• Taxonomy cache: Introducing a caching layer for previously resolved classifications is a reasonable optimisation and is worth carrying forward into any revised design.

Points of concern with the Ask mode proposal

The following aspects of the proposed design warrant further scrutiny before adoption:

• Removal of the message queue: Replacing the queue with a direct synchronous API call introduces risk around system stability. Under concurrent load, the absence of a buffering layer could result in resource exhaustion and cascading failures. A queue provides backpressure, retry semantics, and decoupling that are difficult to replicate in a purely synchronous model.

• Batch prompting: Consolidating multiple product names into a single LLM prompt increases the scope of failure of any individual error. Should the model return a malformed or nonconforming response, the system would need to retry or discard the entire batch, rather than isolating the failure to a single record. This tradeoff between efficiency and fault granularity deserves careful evaluation before adoption.

• Problem analysis: Bob seems to think that a description is a product name in some occasions (like in the "Hierarchical Matching Strategy" section). This could have influenced the decisions it made.

Improving the architecture using Plan mode

According to the IBM Bob Plan mode documentation, you use Plan mode when you need to plan, design, or strategize before implementation. This mode is perfect for breaking down complex problems, creating technical specifications, designing system architecture, or brainstorming solutions before coding.

The following prompt was built from several established prompt-engineering techniques working together. It opens with role assignment, casting the model as a "senior cloud solutions architect" to prime a consistent expert persona and vocabulary. It then imposes an explicit output structure through a numbered, fixed-order list of required deliverables, effectively templating the response so nothing is omitted or reordered. Constraint setting keeps the model on task with clear prohibitions ("do not write code," "do not ask clarifying questions"), while deliberate assumption handling instructs it to state reasonable assumptions rather than stall on missing information. The code-structure table provides few-shot-style grounding, teaching the taxonomy's pattern by worked example rather than abstract description. Finally, the "Quality Bar" section supplies self-evaluation criteria - a rubric the model can check its own output against before finishing. Together these techniques trade open-endedness for precision, steering the model toward a complete, well-formed, and self-contained deliverable.

# Role

You are a senior cloud solutions architect. Your task is to redesign a product classification system end-to-end and produce a complete, self-contained architecture proposal in a single response. This is a planning exercise only — do **not** write implementation code. Do **not** ask clarifying questions; where information is missing, make a reasonable assumption and list it explicitly in an "Assumptions" section.

# Business Context

The client operates a product classification system that maps product names and descriptions to standardized hierarchical classification codes for date- and bean-related commodities.

**Code structure** — 8-digit codes composed of four 2-digit segments, each segment encoding one level of the hierarchy:

| Product | Code |
|---|---|
| Beans | 01000000 |
| Red Beans | 01010000 |
| White Beans | 01020000 |
| Big Red Beans | 01010100 |
| Small Red Beans | 01010200 |
| Spanish Big Red Beans | 01010101 |
| English Big Red Beans | 01010102 |

**Scale:**
- 10^8 product types in the classification taxonomy
- Product catalog of 10^8 distinct items per client dataset
- Theoretical code space of 10^8 combinations
- The taxonomy is finite, known in advance, and changes slowly

# Current Architecture

graph LR
    A@{ shape: circle, label: "User" }
    B@{ shape: rect, label: "Producer" }
    C@{ shape: bow-rect, label: "Message Queue" }
    D@{ shape: procs, label: "Workers"}
    E@{ shape: hex, label: "LLM" }
    F@{ shape: lin-cyl, label: "Cloud Storage" }
    H@{ shape: rect, label: "Preprocessor" }

A --  1. Clean Records --> H
A --  4. Convert Records to Tasks  --> B

subgraph Cloud Provider
    direction BT
    C
    E
    F
    B --  5. Send Tasks  --> C
    C --  6. Send Task  --> D
    D --  7. Get Task Data  ----> F
    D --  8. Get Codes  --> E
    D --  9. Store Result  --> F
    H --  2. Get Records  --> F
    H --  3. Save Cleaned Records  --> F
    subgraph Virtual Machine
        direction RL
        B
        D
        H
    end
end

**Current workflow:** The user manually triggers record cleaning via a Preprocessor running on a VM, which reads raw records from cloud storage and writes cleaned records back. The user then manually triggers a Producer that converts records into tasks pushed to a message queue. Workers consume tasks, fetch record data from cloud storage, call an LLM per task to obtain classification codes, and write results back to cloud storage. A monitoring tool observes the queue.

# Pain Points (must all be addressed)

1. **Manual processing** — every step requires human intervention (triggering preprocessing, triggering task creation), creating bottlenecks and error risk.
2. **Preprocessing overhead** — records must be uploaded to and preprocessed inside the cloud before classification can even begin.
3. **Error susceptibility** — the manual, multi-step workflow introduces human error.
4. **Excessive cost** — cloud storage, per-record LLM API calls, always-on VM, and queue/worker overhead are disproportionate to the task.
5. **Poor client experience** — clients must upload complete datasets to the provider's cloud and wait multiple days for results.

# Design Objectives

- **Fully automated** end-to-end pipeline: zero manual steps between data submission and code delivery.
- **Drastically lower cost**: minimize or eliminate per-record LLM calls, cloud storage footprint, and always-on compute. Critically evaluate whether an LLM is needed at all for this task, given the small, fixed taxonomy — consider deterministic matching, embeddings/vector similarity, classical NLP/fuzzy matching, a cached lookup layer, or a hybrid with LLM fallback only for ambiguous records.
- **Fast turnaround**: results in seconds-to-minutes, not days; favor synchronous or near-real-time processing given the tiny data volume.
- **Better client experience**: clients should not need to upload entire datasets to the provider's cloud — consider client-side/edge processing, an API/SDK, or on-the-fly processing without persistent storage of client data.
- **High accuracy** with built-in validation against the taxonomy (a generated code must always be a valid code in the hierarchy).
- **Simplicity**: remove every component that is not justified at this scale; prefer serverless/managed services or local processing over VMs and queues.

# Constraints & Assumptions

- The classification taxonomy (10^8 codes) is authoritative and available as structured data.
- Input records are product names/descriptions, possibly inconsistent in formatting (hence the current preprocessing step) — preprocessing must be automated, not eliminated silently.
- Client data may be sensitive; minimizing data retention is a plus.
- Stay cloud-provider-agnostic in the design; name component *roles* (e.g., "object storage", "serverless function") and you may give one concrete example per role.
- State any additional assumptions you make.

# Required Deliverables (in this order)

1. **Executive summary** — 3–5 sentences: what changes and why.
2. **Critique of the current architecture** — component by component, identifying what is over-engineered, unnecessary, or misplaced for this workload.
3. **Proposed target architecture** — a Mermaid `graph` diagram (same diagramming style as above, with numbered flow labels) plus a step-by-step description of the new workflow.
4. **Classification strategy** — exactly how codes are generated (matching technique, LLM usage policy if any, validation against the taxonomy, handling of unknown/ambiguous products).
5. **Component decisions table** — for each current component (Preprocessor, Producer, Message Queue, Workers, LLM, Cloud Storage, Monitoring, VM): keep / replace / remove, with one-line justification.
6. **Cost & performance comparison** — qualitative before/after comparison across compute, storage, LLM usage, and turnaround time.
7. **Migration plan** — ordered phases from current to target state with minimal disruption.
8. **Risks & mitigations** — top 3–5 risks of the new design.
9. **Assumptions** — everything you assumed beyond what is stated here.

# Quality Bar

- The proposal must be implementable by an engineering team without further architectural input.
- Every pain point in the "Pain Points" section must be explicitly resolved and traceable to a design decision.
- Prefer the simplest architecture that meets the objectives; justify any component whose monthly cost is non-trivial.
- Output everything in a single response.

Our answer may differ from the answer you will get. In our case, Bob gave an answer consisting of several parts.

Firstly, Bob provided an executive summary of the current architecture, along with some key points that it considered relevant.

Afterwards, Bob proposed a new architecture for the solution, along with an attached step-by-step workflow.

Bob also explained the classification strategy implemented by this workflow.

Bob also provided additional explanations. After this, Bob presented a component decision table to help us better understand the reasoning behind its decisions.

Bob performed a preliminary estimate of the potential cost and performance of the proposed solution and compared it with an estimate of the existing solution.

Bob proposed a migration plan.

Bob also identified several risks associated with this approach.

Lastly, Bob presented several assumptions it made when designing the new architecture as shown in the next two screen captures.

Points of agreement in the plan

The adoption of a Plan mode yields a notable improvement in the quality of Bob's suggestions. Several recommendations are well-aligned with sound architectural thinking:

• Client SDK: The proposal to expose the backend via a dedicated API, or to allow clients to embed it directly into their own backend, is a sound approach that promotes flexibility and clean integration boundaries.
• Query router: Classifying queries by complexity before routing them is a valuable architectural pattern that can meaningfully improve performance and resource allocation.
• Inline record normalization: Introducing automatic preprocessing of text as part of the ingestion pipeline is a practical measure that reduces downstream complexity and improves data consistency.
• Observability and metrics: Instrumenting the system with metrics is a necessary investment for understanding runtime behavior, diagnosing issues, and supporting informed operational decisions.

Points of concern in the plan

Despite the progress above, several concerns remain:

• Removal of the queue: This suggestion is inadvisable and should be reconsidered. A message queue provides critical decoupling between producers and consumers, enables backpressure handling, ensures fault tolerance in the event of downstream failures, and supports asynchronous processing at scale. Removing it would introduce tight coupling and significantly reduce the system's resilience and scalability.
• Excessive output volume: The sheer amount of text generated risks obscuring the primary deliverable, namely the proposed architecture and the technologies underpinning it. The signal-to-noise ratio needs to improve; conciseness should take priority over comprehensiveness.
• Absence of concrete technology recommendations: Bob's suggestions remain at an abstract architectural level and fail to name specific technologies at any stage of the pipeline. Actionable recommendations require concrete tool and framework choices, not just patterns.

Bob in Ask mode as an architectural companion

Even with the improvements from the complex prompt, the suggested architecture cannot be implemented. This is why we propose that you use Bob as a design companion, not as a substitute for architectural planning.

We had some preconceived notions for how to modernize the architecture and which technologies to use. By using Bob in Ask mode, we can challenge those preconceived notions and ask critical questions like:

1. Is this a good choice?
2. What are the alternatives to this technology?
3. Is this aligned with the client needs?

We discovered that the best prompt for comparing architectural options was this one:

I am designing a system with the following architecture:
graph LR
    A@{ shape: circle, label: "Client" }
    B@{ shape: cyl, label: "Queue" }
    C@{ shape: diamond, label: "Complex Description?" }
    D@{ shape: rect, label: "Deterministic Algorithm" }
    E@{ shape: rect, label: "Probabilistic Algorithm" }
    F@{ shape: diamond, label: "Failed?" }
    G@{ shape: cyl, label: "DB Storage" }
    H@{ shape: cyl, label: "Vector Storage" }

subgraph Orchestrator
    B
    subgraph Worker
    C
    D
    E
    F
    end
    H
    G
end

A -- Task --> B
B-- Promise --> A
B --> C
C -- Yes --> E
E --> F
F -- Yes --> D
C -- No --> D
F -- No --> G
D --> G
D <--> H
E <--> H

For the logical block "{LOGICAL_BLOCK}" I have currently selected "{CURRENT_TECHNOLOGY}".

Please suggest 3–5 alternative technologies to "{CURRENT_TECHNOLOGY}" that are similar in characteristics and purpose, evaluating each one in the context of my overall architecture (integration with the other components listed above, operational overhead, ecosystem maturity, and licensing/cost model).

Present the results as a comparison table with the following columns:
| Technology | Type/Category | Key Strengths | Key Weaknesses | Fit with My Architecture | License/Pricing Model | Official Documentation Link |

Include {CURRENT_TECHNOLOGY} as the first row of the table so I can compare alternatives against my current choice. After the table, add a short recommendation: state whether any alternative is clearly better for my case or whether my current choice remains the best fit, and why.

Constraints:
- Only suggest actively maintained technologies.
- Links must point to official documentation, not blog posts.
- If an alternative would require changing another component of my architecture, flag it explicitly.

Our answer may differ from the answer you will get. In our case, Bob gave an answer consisting of several parts.

We compared our workflow orchestrator choice (Conductor OSS) using this prompt in Bob using Ask mode.

Firstly, Bob generated a table comparing several options.

And after, it produced some recommendations.

Even though Bob suggested Temporal as the preferred technology, we would still choose Conductor OSS as it is the only candidate that simultaneously satisfies these requirements:

• Its workers are language agnostic REST polling microservices.
• It uses PostgreSQL as a genuine task queue and workflow persistence store rather than merely a metadata database.
• It requires no additional infrastructure tier such as Kubernetes.

Temporal is the closest alternative and would be a strong contender if the team accepts an SDK-coupling constraint and the higher per action cost profile at volume. Airflow and Prefect each carry structural incompatibilities with the target architecture that go beyond operational preference.

This is also a good illustration of the companion prompt's value: Bob surfaced the right set of alternatives and identified the correct winners. The prompt did its job of generating a structured comparison. The critical evaluation step is what converts the output into a defensible architectural choice.

Summary

In this article, we presented a comprehensive blueprint for modernizing legacy architectures. These principles extend far beyond the specific use case to any scenario where legacy architectures struggle to meet contemporary demands.

After thoroughly testing Bob in both Ask and Plan mode we believe that, when used correctly, it can be a powerful tool for increasing the quality of your solutions while reducing the time spent on it.

As seen throughout this article, Bob (or any LLM IDE for that matter) cannot replace critical thinking. Had we gone with Bob's proposed solution outright, we would have ended up with an improved but ultimately incomplete result. That said, using Bob to accelerate the search and comparison workflows (arguably the most time-consuming part of system design) can substantially cut down solution modernisation time.

Regardless of these capabilities, it is always worth continuing to learn independently of LLMs and staying curious about emerging technologies. LLMs tend to favour tried-and-tested solutions, which can inadvertently obscure newer, potentially better alternatives.