image

Bootcamps ilimitados + curso de inglês para sempre

80
%OFF
Alexandre Spindola
Alexandre Spindola23/07/2025 09:07
Compartilhe
Randstad - Análise de DadosRecomendados para vocêRandstad - Análise de Dados

Building a Multilingual Digital Library: A JAMstack Journey with Automated Translation

    Building a Multilingual Digital Library: A JAMstack Journey with Automated Translation

    As a thesis project (TCC), I recently completed the development of a comprehensive multilingual digital library that leverages modern JAMstack architecture combined with automated translation workflows. This project demonstrates how to build scalable, performance-oriented web applications while solving real-world accessibility challenges through intelligent automation.

    🎯 Project Overview

    The Multilingual Gnostic Digital Library is a static site built with Astro that automatically translates spiritual and philosophical texts into 110 languages using Google Gemini AI. The system integrates Strapi as a headless CMS with n8n for workflow automation, creating a powerful content management and translation pipeline.

    🌐 Live Demo: https://books.gnosisdeutschland.org

    The main objective is to democratize access to Gnostic knowledge by eliminating language barriers through reliable automated translations, offering an accessible and efficient solution for multilingual content management.

    🏗️ Architecture & Tech Stack

    Frontend:

    • Astro with Starlight for documentation-style presentation
    • Islands Architecture for optimized JavaScript delivery
    • Tailwind CSS for responsive design with utility-first approach
    • TypeScript for type safety
    • Static Site Generation for optimal performance
    • Light/dark theme toggle with user preference persistence
    • Integrated search functionality with content indexing

    Backend & CMS:

    • Strapi headless CMS for content management (deployed on Railway)
    • PostgreSQL for data persistence
    • Docker for containerization
    • Relational structure: Authors → Books → Categories → Chapters
    • Locales system to associate translated versions with original content

    Automation & Translation:

    • n8n for workflow orchestration (configured with 5 workers for parallel processing)
    • Google Gemini AI for intelligent translation
    • Custom TypeScript scripts for content generation
    • Training system to improve translation quality
    • Two main workflows: Books (titles/slugs) and Chapters (content/formatting)

    🚀 Key Features

    1. Automated Translation Pipeline

    The system supports translation into 110 languages with intelligent context preservation. The average processing time for translating a chapter into all languages is approximately 60 minutes (about 30 seconds per language):

    // Translation workflow configuration in n8n
    const translationPrompt = `
    Translate the title field below to specified language and country. 
    
    Training data:
    Input (original in spanish): Book Title Example
    English translation: English Book Title
    Portuguese translation: Portuguese Book Title
    Italian translation: Italian Book Title
    
    IMPORTANT: You should keep the text capitalised, like the example Input.
    Return ONLY the exactly translated text, no other words.
    
    Input to translate: Book Title to Translate
    `;
    

    2. Dynamic Content Generation

    The project includes sophisticated TypeScript scripts that automatically generate localized content and convert CMS data into reusable MDX files:

    // generateBooks.ts - Automatic book page generation
    async function generateIndexMarkdown(book: Book, locale: string): string {
    const coverImage = book.cover_image?.formats?.small?.url ?? "";
    const title = book.title;
    
    // Load translations for the specific locale
    const translations = loadTranslations(locale);
    const readBookText = translations.books?.readBook || 'Read book';
    
    return `---
    title: "${title}"
    lang: ${locale}
    template: splash
    hero:
    author: Samael Aun Weor
    image:
      file: ${coverImage}
    actions:
      - text: ${readBookText}
        link: prefacio
        icon: right-arrow
    sidebar:
    label: Intro
    order: -1
    ---`;
    }
    

    3. Intelligent Caching & Performance

    The system implements locale-specific caching for optimal build performance with language-specific timestamps and incremental updates:

    // Smart caching system for incremental updates
    function getLastBuildTimestamp(locale: string): string {
    const cacheFile = path.resolve(CACHE_DIR, `${locale}.timestamp`);
    
    if (!fs.existsSync(cacheFile)) {
      return "2020-01-01T00:00:00.000Z";
    }
    
    return fs.readFileSync(cacheFile, "utf-8").trim();
    }
    
    async function fetchChapters(bookId: string, locale: string, bookData: Book): Promise<Chapter[]> {
    const lastBuildTimestamp = getLastBuildTimestamp(locale);
    const searchLocale = locale === 'pt' ? 'pt-BR' : locale;
    
    // Only fetch updated content since last build
    const response = await axios.get(
      `${STRAPI_URL}/api/chapters?filters[updatedAt][$gt]=${lastBuildTimestamp}&locale=${searchLocale}`
    );
    }
    

    🐳 Containerized Development Environment

    The project includes a complete Docker setup with n8n for workflow automation:

    # Custom n8n container with additional dependencies
    FROM n8nio/n8n:next
    
    USER root
    
    RUN npm install -g node-fetch@2 moment moment-timezone lodash uuid validator date-fns jsonpath joi form-data
    
    USER node
    
    # docker-compose.yaml - Production-ready setup
    services:
    postgres:
      image: postgres:17
      environment:
        - POSTGRES_USER
        - POSTGRES_PASSWORD
        - POSTGRES_DB
      healthcheck:
        test: ['CMD-SHELL', 'pg_isready -h localhost -U ${POSTGRES_USER} -d ${POSTGRES_DB}']
        interval: 5s
        timeout: 5s
        retries: 10
    
    n8n:
      build: .
      command: start
      ports:
        - "127.0.0.1:5678:5678"
      environment:
        - N8N_BASIC_AUTH_ACTIVE=true
        - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY}
        - EXECUTIONS_MODE=queue
        - QUEUE_BULL_REDIS_HOST=redis
    
    n8n-worker-1:
      build: .
      command: worker --concurrency=20
      depends_on:
        - n8n
    

    🌍 Astro Configuration for Multilingual Support

    The Astro configuration showcases advanced internationalization setup:

    // astro.config.mjs
    export default defineConfig({
    output: "static",
    site: "https://books.gnosisdeutschland.org",
    integrations: [
      sitemap({
        changefreq: "weekly",
        priority: 0.7,
        lastmod: new Date(),
      }),
      starlight({
        prerender: true,
        title: "GNOSIS",
        logo: {
          src: "./src/img/favicon.png",
        },
        customCss: [
          "@fontsource/lato/400.css",
          "@fontsource/lato/700.css",
          "./src/styles/css/tailwind.css",
          "./src/styles/css/base.css",
          "./src/styles/css/custom.css",
        ],
        // Dynamic sidebar imports for different books
        sidebar: [
          librosTranslations,
          educacionFundamentalTranslations,
          laGranRebelionTranslations,
          tratadoDePsicologiaRevolucionariaTranslations,
          tratadoEsotericoDeAstrologiaHermetica,
        ]
      }),
      tailwind()
    ]
    });
    

    🚀 Automated Deployment Pipeline

    The deployment process is fully automated with performance optimizations:

    #!/bin/bash
    # deploy.sh - Automated deployment script
    
    START_TIME=$(date +%s)
    echo "🚀 Starting deployment process..."
    
    # Generate content dynamically
    echo "📚 Generating books..."
    npx tsx scripts/generateBooks.ts
    
    echo "📚 Generating chapters..."
    npx tsx scripts/generateChapters.ts
    
    if [ $? -eq 0 ]; then
      echo "🧹 Cleaning caches..."
      rm -rf dist .astro
    
      echo "🛠️ Building project..."
      NODE_OPTIONS=--max_old_space_size=4096 pnpm build
    
      if [ $? -eq 0 ]; then
          echo "📤 Uploading to Netlify..."
          netlify deploy --prod --dir=dist
      fi
    fi
    
    END_TIME=$(date +%s)
    DURATION=$((END_TIME - START_TIME))
    echo "✅ Deployment complete in ${DURATION}s!"
    

    📊 Performance Metrics

    The digital library demonstrates exceptional performance with real-world results:

    Desktop Performance (Google PageSpeed Insights: 100/100)

    • First Contentful Paint (FCP): 299ms
    • Largest Contentful Paint (LCP): 465ms
    • Total Blocking Time (TBT): 0ms
    • Cumulative Layout Shift (CLS): 0
    • Speed Index: 620ms

    Mobile Performance (Google PageSpeed Insights: 96/100)

    • First Contentful Paint (FCP): 1,576ms
    • Largest Contentful Paint (LCP): 2,112ms
    • Total Blocking Time (TBT): 0ms
    • Cumulative Layout Shift (CLS): 0
    • Speed Index: 4,130ms

    Infrastructure & Deployment

    • Platform: Netlify with global CDN distribution
    • SSL: Automatic certificate management
    • Cache: Selective cache invalidation for incremental updates
    • Uptime: 99.9% availability

    🔄 Translation Automation Workflows

    The translation process operates through two specialized n8n workflows:

    1. Books Workflow: Translates titles and automatically generates slugs
    2. Chapters Workflow: Translates titles, content, and generates slugs while preserving Markdown formatting

    Both workflows implement a training system that uses previous translations as context to improve quality and consistency across languages.

    🎯 Key Achievements & Learnings

    Performance Optimization

    • Static Site Generation for lightning-fast loading
    • Incremental builds with smart caching
    • CDN deployment through Netlify
    • Image optimization with responsive formats

    Scalability Solutions

    • Microservices architecture with Docker
    • Horizontal scaling with multiple n8n workers
    • Database optimization with PostgreSQL
    • Queue-based processing for translation workflows

    Developer Experience

    • TypeScript throughout the stack for type safety
    • Hot module replacement for rapid development
    • Automated testing and quality assurance
    • Version control with Git hooks and CI/CD

    🔮 Future Enhancements

    1. Real-time translation monitoring dashboard
    2. A/B testing for translation quality
    3. Voice synthesis for audio content
    4. Progressive Web App features with offline capabilities via Service Workers
    5. Advanced SEO optimization for multilingual content
    6. Pre-fetch techniques for instant navigation
    7. Incremental static regeneration strategies
    8. Real-time synchronization mechanisms between CMS and frontend

    📊 Impact & Results

    • 110 languages supported automatically
    • Sub-second page loads through static generation
    • 99.9% uptime with Netlify hosting
    • Scalable architecture ready for thousands of books
    • Zero manual translation required
    • 60-minute average processing time for complete chapter translation
    • Perfect accessibility scores with responsive design

    🛠️ Technical Implementation

    The complete implementation showcases:

    • Modern JAMstack architecture with Astro's islands hydration
    • Headless CMS integration with Strapi on Railway
    • Automated CI/CD pipeline with Netlify deployment
    • Intelligent translation workflows using Google Gemini AI
    • Performance-first approach achieving perfect Lighthouse scores

    🔗 Explore the Project:

    This project demonstrates how modern JAMstack architecture can solve complex real-world problems while maintaining excellent performance and developer experience. The combination of Astro's static generation, Strapi's flexibility, and n8n's automation capabilities creates a powerful foundation for content-driven applications.

    The complete source code and documentation are available, showcasing best practices in TypeScript, Docker, and modern web development workflows.

    What challenges have you faced when building multilingual applications? Share your experiences in the comments below!

    Compartilhe
    Recomendados para você
    GFT Start #7 .NET
    GFT Start #7 - Java
    Deal Group - AI Centric .NET
    Comentários (1)
    DIO Community
    DIO Community - 23/07/2025 09:56

    Projeto impressionante, Alexandre. A forma como você combinou arquitetura JAMstack, automação com n8n e tradução inteligente com IA para construir uma biblioteca multilíngue é um excelente exemplo de aplicação prática e escalável de tecnologias modernas para resolver um problema real de acessibilidade e distribuição de conhecimento.

    A estrutura do projeto ficou extremamente robusta e ao mesmo tempo elegante, do uso do Astro com Starlight para performance e acessibilidade, ao pipeline automatizado que traduz, versiona e publica conteúdo com mínima intervenção manual. A integração entre Strapi, PostgreSQL, TypeScript e n8n mostra um domínio técnico admirável, e os ganhos de desempenho relatados reforçam a eficiência dessa stack.

    Pensando na maturidade da sua arquitetura, quais estratégias você considera mais eficazes para garantir qualidade linguística e fidelidade semântica ao escalar para novos idiomas em cenários de produção mais sensíveis, como textos técnicos, jurídicos ou religiosos?

    Recomendados para vocêRandstad - Análise de Dados