Introduction

Miden, a modern Issuer Processor and Banking Stack provider for businesses, has been experiencing rapid growth, leading to an explosion in data volume, with over 300 million traces and logs generated daily. This scale, coupled with their stringent requirements for on-premise data sovereignty and the need for granular access control across more than 20 microservices, pushed their existing CloudWatch-based observability solution to its limits. Miden turned to the Osuite Observability Suite, an on-premise solution, to gain deep insights, enhance security, and manage their rapidly growing telemetry data. The implementation resulted in robust, scalable observability, the identification of critical performance bottlenecks, significant cost reductions, and fortified security through precise access control via Keycloak.

The Customer: Miden, Building the banking stack of the future

Midenis a fintech infrastructure company specializing in providing a modern issuer processor and comprehensive banking stack platform primarily for businesses and financial institutions in Sub-Saharan Africa. Their platform enables organizations to quickly launch and manage payment programs, including virtual card issuance, corporate cards, wallet services, bill payments, and automated financial workflows. Miden focuses on reducing integration time, minimizing card fraud, and improving system stability while offering bank-grade security and compliance.

As a company at the forefront of fintech innovation, Miden needs an observability solution that could keep pace with their rapid expansion and evolving technological needs.

The Challenge: Overcoming observability hurdles in a high-growth, secure fintech environment

Miden's reliance on CloudWatch presented several critical limitations as their operations scaled:

• Data Sovereignty and Security Concerns: They aimed to minimize broad AWS console access for enhanced security.
• Querying Inflexibility: CloudWatch lacked the flexibility Miden required to query arbitrary data within their logs, hindering their ability to gain specific, actionable insights
• Absence of Distributed Tracing: With over 20 microservices, understanding endpoint performance and identifying bottlenecks across their distributed architecture was nearly impossible without comprehensive distributed tracing capabilities.
• Operational Inefficiencies: Troubleshooting was a slow and cumbersome process, as teams had to manually sift through undifferentiated logs from all microservices.
• Security and Compliance Gaps: The previous setup did not allow for granular, team-specific log access. Granting broad access to all logs across all teams posed significant security risks and complicated compliance efforts.

Why Osuite Observability Suite?

Miden selected the Osuite Observability Suite after careful consideration, driven by several key differentiators:

• On-Premise Deployment: Osuite’s ability to be deployed within Miden’s own cloud environment was paramount, directly addressing their data sovereignty and security requirements.
• Predictable and Lower Costs: Unlike incumbent solutions with ingestion-based pricing, Osuite offered a flat-rate model. This provided predictable costs and is least 10 times lower than alternatives like Datadog or Splunk, a crucial factor given their 300M+ daily trace and logs volume.
• Granular Access Control: The seamless integration with Keycloak allowed Miden to implement fine-grained, application-level access control to the logs. This was essential ensuring teams only accessed data relevant to their roles.
• Comprehensive Feature Set: Osuite provided robust capabilities for high-volume data ingestion, distributed tracing across microservices, and flexible, powerful querying, Application Performance Monitoring, Real time alerts – all essential for Miden's complex environment.

The Solution: Rapid on-premise deployment with Osuite

The deployment of the Osuite Observability Suite within Miden's AWS environment was remarkably swift and efficient:

• Streamlined Installation: The process began with creating an IAM user with the necessary permissions. The Osuite installer then completed the core setup in approximately one hour.
• Effortless Microservice Integration: Leveraging the industry-standard OpenTelemetry, Miden integrated their 20+ microservices with Osuite, a process that also took about an hour.
• Robust Access Control Configuration: Keycloak was configured to manage granular access to all components of the observability suite, with control handed over to Miden's team.
• Swift Implementation Timeline: The initial setup was achieved in roughly one day, with full integration of all their services completed within a week.
• Smooth Transition: The implementation faced no significant challenges. Miden opted for a fresh start with the new system, so no data migration from CloudWatch was required. Comprehensive "how-to" guides and dedicated support from the Osuite team ensured a seamless onboarding experience.

The Results: Enhanced performance, drastic cost reduction, and unprecedented control

The adoption of the Osuite Observability Suite delivered transformative results for Miden:

• Scalability and Stability: The solution comfortably handles over 300 million events(traces+logs) per day and sustains system stability even during peak loads of tens of millions of events per minute (with an average load of around 100,000 requests per minute). This reliability is indispensable for a payment processor.
• Actionable Insights and Proactive Optimisation:
  • Bottleneck Identification: Osuite enabled Miden to clearly identify system load distribution, pinpoint performance bottlenecks, and discover their slowest endpoints.
  • Critical Incident Resolution & Cost Savings: In a standout success, Osuite helped identify a single endpoint that was being hit approximately 2 million times per minute during peak periods. This was overwhelming their systems and significantly inflating costs. Using Osuite, Miden traced the activity to a specific customer, addressed the excessive usage, and successfully implemented rate limiting. This intervention drastically reduced system load and delivered substantial cost savings.
• Significant Cost Efficiency: Beyond the direct savings from optimising system load, Miden benefits from Osuite’s flat-rate pricing, ensuring a significantly lower Total Cost of Ownership (TCO) compared to ingestion-based observability platforms.
• Enhanced Security and Compliance: The granular access control, managed through Keycloak, ensures that development and operations teams can only view logs pertinent to their specific microservices or areas of responsibility, bolstering Miden's security posture and simplifying compliance.

(Miden anticipates further improvements in key operational metrics like Mean Time to Resolution (MTTR) as they continue to leverage Osuite's capabilities.)

Future Plans: Scaling with a trusted Observability partner

As Miden continues its impressive growth trajectory and plans to expand its microservices architecture, the Osuite Observability Suite is positioned as a critical component of its technology stack. They rely on Osuite to provide reliable, scalable, and insightful observability, ensuring their platform remains robust, performant, and secure as they build the future of business banking.