High-Performance Search Cluster for meindm.at
From 2013 to 2019, I architected and maintained the Apache Solr search infrastructure powering meindm.at—dm-drogeriemarkt’s digital platform. Over six years, I evolved this system from Solr 4.1 to 7.x, building a sophisticated distributed search cluster that delivered intelligent product discovery while offloading expensive database queries from Oracle systems.
The Search Challenge.
As meindm.at grew to serve millions of customers across multiple platforms (web shop, mobile apps, kiosks), the search functionality became a critical bottleneck:
- Database Load: Search queries against Oracle database were expensive and impacted overall system performance
- User Expectations: Customers expected Google-like search with instant results, autocomplete, and intelligent suggestions
- Product Catalog Complexity: Thousands of SKUs across beauty, healthcare, and household categories required nuanced search
- Language Support: Multi-country operations demanded language-specific search optimization
- Scale Requirements: Had to handle concurrent searches from web, mobile, and kiosk platforms
The Solution: Distributed Search Architecture
I designed and implemented a comprehensive Apache Solr cluster that transformed search from a performance liability into a competitive advantage.
Technical Architecture
Distributed Search Cluster Built a horizontally scalable Solr cluster with:
- Up to 12 distributed cores for different data types and use cases
- ZooKeeper cluster for distributed coordination and configuration management
- Load-balanced architecture ensuring high availability and query distribution
- Managed and unmanaged core configurations for different operational requirements
Continuous Evolution Maintained the platform through six years of Solr releases:
- Started with Solr 4.1 (2013)
- Implemented continuous integration for seamless upgrades
- Evolved architecture through major version changes
- Ended with Solr 7.x (2019)
- Ensured zero downtime during upgrades
Intelligent Search Features
Natural Language Search Implemented sophisticated search capabilities that understood user intent:
- Synonym Search: Integrated OpenThesaurus for German language synonyms, so searching “Creme” also finds “Lotion” and related terms
- Stemming: Language-specific word reduction (e.g., “running” → “run”) for better recall
- Autocomplete: Real-time suggestions as users type, guiding them to popular searches
- Spellcheck: Graceful handling of typos with “Did you mean…?” suggestions
- Faceted Navigation: Dynamic filtering by category, brand, price, and attributes
Domain-Specific Intelligence Tuned search specifically for beauty and healthcare retail:
- Custom thesaurus for beauty terminology
- Product attribute boosting (brand names, featured products)
- Seasonal relevance adjustments
- Promotional product prioritization
Performance Optimization
Database Offloading The primary goal was reducing Oracle database load:
- Indexed product catalog, attributes, and metadata in Solr
- Directed search queries to Solr cluster instead of database
- Reduced database query load by orders of magnitude
- Freed database resources for transactional operations
Query Performance Optimized for sub-second response times:
- Distributed query processing across cluster nodes
- Intelligent caching strategies
- Index optimization and compaction
- Query result pagination and performance tuning
RESTful JSON API Developed a clean REST API for search services:
- JSON responses for easy integration with web and mobile clients
- jQuery-based frontend integration
- Consistent API across all platforms
- Performance monitoring and query analytics
Infrastructure & Operations
Cluster Management Took comprehensive ownership of:
- ZooKeeper ensemble configuration and monitoring
- Core management and distribution strategies
- Index replication and failover configuration
- Performance tuning and resource optimization
Continuous Integration Established practices for safe, continuous evolution:
- Testing frameworks for search quality
- Blue-green deployment strategies for upgrades
- Rollback procedures for failed updates
- Configuration version control
Hardware & Procurement Responsible for:
- NetApp storage configuration for index data
- Linux server deployment and optimization
- Capacity planning and scaling decisions
- Infrastructure cost optimization
Technology Stack
Search Platform
- Search Engine: Apache Solr (4.1 → 7.x)
- Coordination: Apache ZooKeeper cluster
- Distribution: Up to 12 distributed cores
Integration
- API: REST Web Services
- Data Format: JSON
- Frontend: jQuery
- Web Server: Apache HTTP Server
- App Server: Apache Tomcat
Backend
- Language: Java
- Database: Oracle Database
- Storage: NetApp Enterprise Storage
- Platform: Linux
Tools
- Testing: Apache JMeter for performance validation
- Project Management: Jira, Confluence
- Monitoring: Custom analytics and query logging
Key Achievements
Performance Impact
- Reduced Oracle load by 80%+ through search query offloading
- Sub-second search response times even under peak load
- Concurrent query handling across web, mobile, and kiosk platforms
- Horizontal scalability through distributed architecture
Search Quality
- Intelligent autocomplete guiding users to relevant products
- Synonym expansion improving search recall for German language queries
- Spellcheck tolerance handling typos and variations gracefully
- Faceted navigation enabling intuitive product discovery
Operational Excellence
- Six years of continuous operation with high availability
- Zero-downtime upgrades from Solr 4.1 to 7.x
- Proactive maintenance and performance optimization
- Scalable architecture growing with business needs
Evolution & Continuous Improvement
The six-year engagement demonstrated commitment to continuous improvement:
2013 (Solr 4.1)
- Initial cluster deployment
- Basic search with synonym support
- Foundation for future growth
2014-2016
- Enhanced autocomplete and suggestions
- Advanced faceting capabilities
- Performance tuning and optimization
2017-2019 (Solr 7.x)
- Modern Solr architecture
- Enhanced distributed capabilities
- Improved monitoring and management
Impact on meindm.at
The Solr search infrastructure became foundational to the digital platform:
- Enabled Product Discovery: Helped millions of customers find products across the catalog
- Improved User Experience: Fast, intelligent search matching modern user expectations
- Reduced Infrastructure Costs: Offloading Oracle reduced database licensing and hardware needs
- Supported Growth: Scaled seamlessly as product catalog and user base expanded
- Powered Multiple Platforms: Single search infrastructure serving web, mobile, and kiosk
This project showcased the ability to architect and evolve enterprise search infrastructure over multiple years, maintaining operational excellence while continuously improving capabilities and keeping pace with technology evolution.
Duration: 2013 – 2019 (6 years) Role: Software & System Architect Location: Austria Client: dm-drogeriemarkt / Cards & Systems Platform: meindm.at (web shop, mobile apps, kiosks)
