Developer Onboarding Guide - InnoQualis EQMS

This comprehensive guide provides step-by-step instructions for developers joining the InnoQualis Electronic Quality Management System (EQMS) project. It covers environment setup, development workflows, testing procedures, and contribution guidelines.

Last Updated: January 2025
Version: Phase 8 In Progress (Documentation Consolidation Complete)
Status: Production Ready

Quick Start

For the fastest setup, use the live reloading development environment with make dev-live-up. This provides instant code changes without container rebuilds.

Prerequisites

System Requirements

• Operating System: macOS, Linux, or Windows with WSL2
• Memory: 8GB RAM minimum, 16GB recommended
• Storage: 10GB free space for dependencies and Docker images
• Network: Stable internet connection for dependency downloads

Required Software

• Node.js: Version 20+ (use nvm for version management)
• Python: Version 3.11+ (use pyenv for version management)
• Docker: Version 24+ with Docker Compose
• Git: Version 2.30+
• pnpm: Node.js package manager (install via npm: npm install -g pnpm)
• Poetry: Python dependency manager (primary) - install via pip: pip install poetry
• uv: Fast Python package manager (alternative) - install via: curl -LsSf https://astral.sh/uv/install.sh | sh

Optional but Recommended

• VS Code: With Python, TypeScript, and Docker extensions
• Postman: For API testing
• DBeaver: For database management
• GitHub CLI: For repository management

Quick Start (5 Minutes)

1. Clone the Repository

git clone https://github.com/your-org/innoqualis-eqms.git
cd innoqualis-eqms

2. Set Up Environment Variables

# Copy environment template
cp backend/env.example backend/.env

# Edit backend/.env with your configuration
# Key variables to set:
# - DATABASE_URL (default: postgresql://user:password@localhost:5432/eqms)
# - SECRET_KEY (generate a secure random string)
# - OPENAI_API_KEY (if using AI features)

3. Start Development Environment

# Start all services with Docker Compose
make dev-live-up

# Or start individual services
make local-up

4. Verify Installation

# Check backend health
curl http://localhost:8000/health

# Check frontend
open http://localhost:3000

Detailed Setup Instructions

Backend Setup

1. Python Environment Setup

cd backend

# Install Poetry (if not already installed)
curl -sSL https://install.python-poetry.org | python3 -

# Install dependencies
poetry install

# Activate virtual environment
poetry shell

# Verify installation
python --version  # Should show Python 3.11+
poetry --version  # Should show Poetry version

2. Database Setup

# Start PostgreSQL with Docker
docker-compose up -d postgres

# Run database migrations
make migrate

# Seed database with test data
make seed

# Verify database connection
poetry run python -c "from app.database import get_db; print('Database connected successfully')"

3. Backend Development Server

# Start backend server
poetry run python run.py

# Or use the development script
make local-backend

Backend will be available at: http://localhost:8000
API Documentation: http://localhost:8000/docs

Frontend Setup

1. Node.js Environment Setup

cd frontend

# Install pnpm (if not already installed)
npm install -g pnpm

# Install dependencies
pnpm install

# Verify installation
node --version  # Should show Node.js 20+
pnpm --version  # Should show pnpm version

2. Frontend Development Server

# Start frontend development server
pnpm dev

# Or use the development script
make local-frontend

Frontend will be available at: http://localhost:3000

Docker Development Setup

1. Full Stack with Docker Compose

# Start all services (recommended for development)
make dev-live-up

# View logs
make dev-live-logs

# Stop services
make dev-live-down

2. Individual Service Management

# Start specific services
docker-compose up -d postgres redis

# Rebuild containers
make dev-live-rebuild

# Clean up
docker-compose down -v

Development Workflows

Daily Development Workflow

1. Start Your Day

# Pull latest changes
git pull origin main

# Start development environment
make dev-live-up

# Run tests to ensure everything works
make test-backend
make test-frontend

2. Feature Development

# Create feature branch
git checkout -b feature/your-feature-name

# Make changes
# ... edit code ...

# Run tests frequently
make test-backend
make test-frontend

# Commit changes
git add .
git commit -m "feat: add your feature description"

3. End of Day

# Push changes
git push origin feature/your-feature-name

# Stop development environment
make dev-live-down

Testing Workflows

Backend Testing

# Run all backend tests
cd backend && poetry run pytest -q .

# Run specific test file
poetry run pytest tests/test_documents.py -v

# Run tests with coverage
poetry run pytest --cov=app --cov-report=html

# Run contract tests
poetry run pytest tests/test_contract.py

Frontend Testing

# Run unit tests
cd frontend && pnpm test

# Run E2E tests
pnpm test:e2e

# Run specific E2E test
pnpm playwright test --grep "document lifecycle"

# Run tests with coverage
pnpm test:coverage

Integration Testing

# Run full integration test suite
make test-integration

# Run specific integration tests
cd backend && poetry run pytest tests/integration/ -v

Code Quality Workflows

Backend Code Quality

# Format code
cd backend && poetry run black .
poetry run isort .

# Lint code
poetry run flake8 .

# Type checking
poetry run mypy app/

Frontend Code Quality

# Format code
cd frontend && pnpm lint:fix

# Lint code
pnpm lint

# Type checking
pnpm type-check

Project Structure

Backend Structure

backend/
├── app/
│   ├── routers/          # API route handlers
│   ├── models.py         # Database models
│   ├── schemas.py        # Pydantic schemas
│   ├── services/         # Business logic
│   ├── utils/           # Utility functions
│   └── main.py          # FastAPI application
├── tests/               # Test files
├── alembic/            # Database migrations
├── scripts/            # Utility scripts
└── requirements.txt    # Python dependencies

Frontend Structure

frontend/
├── components/         # React components
├── pages/             # Next.js pages
├── hooks/             # Custom React hooks
├── lib/               # Utility libraries
├── __tests__/         # Test files
├── e2e-tests/         # E2E test files
└── public/            # Static assets

Key Files to Know

• AGENTS.md: Comprehensive development guidelines
• Makefile: Common development commands
• docker-compose.yml: Service orchestration
• contracts/openapi.yaml: API specification
• docs/: Project documentation
• docs/site/content/dev/development-history.md: Development history and major milestones
• docs/site/content/dev/user-journey-validation.md: Current E2E validation scenario

Major System Features

Configurable Document Workflow System

The system includes a comprehensive workflow system for documents:

• Document Types: Admin-configurable document types with default workflow templates
• Workflow Templates: Customizable workflow templates with states, transitions, and approval rules
• State-Based Approvals: Role-based and user-specific approval rules per workflow state
• Auto-Transitions: Automatic state transitions when approval requirements are met
• Workflow History: Complete audit trail of all workflow state transitions

Key Files:

• Models: backend/app/models.py (DocumentType, DocumentWorkflowTemplate, DocumentWorkflowState, etc.)
• Service: backend/app/services/document_workflow_service.py
• API: backend/app/routers/document_types.py, backend/app/routers/workflow_templates.py
• Migration: backend/backend/versions/20250115_01_create_workflow_system.py

Dynamic Form Builders

The system supports configurable, template-based forms for both Deviations and CAPAs:

Deviation Form Builder (/admin/deviation-form-templates):

• Form Templates: Admin-configurable form templates with sections and fields
• Field Types: Support for text, textarea, select, document_picker, capa_picker, image, file uploads
• AI Integration: AI-powered field suggestions (e.g., severity classification with confidence scores)
• Related Items: Link deviations to multiple documents and CAPAs via picker fields
• Attachment Support: Image and file uploads with captions and descriptions

CAPA Form Builder (/admin/capa-form-templates):

• Form Templates: Admin-configurable CAPA form templates with sections and fields
• Field Types: Support for text, textarea, select, document_picker, deviation_picker, image, file uploads
• Related Items: Link CAPAs to multiple documents and deviations via picker fields
• Attachment Support: Image and file uploads with captions and descriptions

Key Files - Deviation Forms:

• Models: backend/app/models.py (DeviationFormTemplate, DeviationFormSection, DeviationFormField, DeviationFormData)
• API: backend/app/routers/deviation_form_templates.py
• Enhanced API: backend/app/routers/deviations.py (enhanced creation endpoint)
• Migration: backend/backend/versions/20250128_04_deviation_form_builder.py

Key Files - CAPA Forms:

• Models: backend/app/models.py (CAPAFormTemplate, CAPAFormSection, CAPAFormField, CAPAFormData)
• API: backend/app/routers/capa_form_templates.py
• Enhanced API: backend/app/routers/capa.py (enhanced creation endpoint)
• Migration: backend/backend/versions/20250129_01_capa_form_builder.py

Shared Components:

• frontend/components/forms/TemplateSelector.tsx - Reusable template selector
• frontend/components/forms/FieldTypeSelector.tsx - Visual field type selector with icons
• frontend/components/deviations/DynamicFormField.tsx - Dynamic field renderer (enhanced with deviation_picker)

Common Development Tasks

Adding a New API Endpoint

1. Backend Implementation

# Create new router
touch backend/app/routers/new_module.py

# Add to main.py
# ... edit backend/app/main.py ...

# Create database model
# ... edit backend/app/models.py ...

# Create Pydantic schemas
# ... edit backend/app/schemas.py ...

# Write tests
touch backend/tests/test_new_module.py

2. Frontend Integration

# Generate TypeScript types
cd frontend && pnpm run gen:api:types

# Create React component
touch frontend/components/NewModule.tsx

# Create page
touch frontend/pages/new-module.tsx

# Write tests
touch frontend/__tests__/NewModule.test.tsx

Database Management

Critical Rule: Always Add Tables Through Migrations

NEVER manually create tables without creating/updating a migration file.

Migration Best Practices

1. Check if Table Exists Before Creating

All migration upgrade functions should check if tables already exist:

def upgrade():
    # Check if table already exists
    from sqlalchemy import inspect
    conn = op.get_bind()
    inspector = inspect(conn)
    tables = inspector.get_table_names()
    
    if "my_table" not in tables:
        op.create_table(...)
    else:
        print("INFO: my_table already exists, skipping creation")

2. Check if Indexes Exist Before Creating

Similarly for indexes:

def upgrade():
    if "my_table" not in tables:
        # Create table
        pass
    else:
        indexes = [idx['name'] for idx in inspector.get_indexes("my_table")]
        
        if "my_index" not in indexes:
            op.create_index("my_index", "my_table", ["column"])

3. Sequence of Operations

When adding a new table:

1. Create or update a migration file in backend/backend/versions/
2. Use Alembic to manage schema changes
3. If manual creation was necessary (emergency fix), immediately update the migration to make it idempotent

Migration Workflow

Creating a New Migration

# Create a new migration
cd backend
poetry run alembic revision -m "add_new_feature"

# Edit the generated migration file in backend/backend/versions/
# Add existence checks for tables and indexes

# Apply migration
poetry run alembic upgrade head

Testing Migrations

# Check current migration state
docker compose exec backend alembic current

# See migration history
docker compose exec backend alembic history

# Run migrations
docker compose exec backend alembic upgrade head

# Rollback one migration
docker compose exec backend alembic downgrade -1

Migration Order

Migrations are applied in order based on their revision numbers and down_revision references. Always check existing migrations before creating new ones.

Key Migrations

Key database migrations implementing major features:

Migration File	Feature	Date
20250115_01_create_workflow_system.py	Configurable document workflow system	January 2025
20250128_03_capa_deviation_associations.py	CAPA-deviation many-to-many associations	January 2025
20250128_04_deviation_form_builder.py	Dynamic deviation form builder	January 2025
20250129_01_capa_form_builder.py	Dynamic CAPA form builder	January 2025
20250128_05_ai_classification_fields.py	AI-powered severity classification	January 2025

For complete development history, see Development History.

Database Changes

1. Create Migration

Follow the Migration Best Practices above. Always include idempotent checks in your migration file.

cd backend

# Create new migration
poetry run alembic revision --autogenerate -m "Add new table"

# Edit migration file to add existence checks
# ... edit backend/backend/versions/[migration_file].py ...

# Apply migration
poetry run alembic upgrade head

2. Update Models and Schemas

# Update models.py
# ... edit backend/app/models.py ...

# Update schemas.py
# ... edit backend/app/schemas.py ...

# Regenerate API types
cd frontend && pnpm run gen:api:types

3. Verify Migration

After creating and applying a migration:

• Verify tables/columns exist in database
• Test that models can access new fields
• Update related API endpoints if needed
• Update tests to cover new schema changes

Working with Workflow System

Understanding Workflow Templates

Workflow templates define the lifecycle of documents:

1. Document Types: Define categories of documents (SOP, Policy, etc.)
2. Workflow Templates: Define states, transitions, and approval rules
3. Workflow Instances: Track individual documents through their workflow

Creating Workflow Templates

# Workflow templates are managed via API
# See backend/app/routers/workflow_templates.py

# Or use the admin UI:
# Navigate to /admin/workflow-templates

Document Upload with Workflow

When uploading a document:

1. Select document type (defaults to available types)
2. Select workflow template (or use default for document type)
3. System initializes workflow instance
4. Document follows workflow-defined states

See docs/implementation-status.md for detailed workflow system documentation.

Working with Form Builders

Understanding Form Templates

Form templates define the structure of deviation and CAPA forms:

1. Templates: Top-level container with sections
2. Sections: Grouped fields (e.g., "Basic Information", "Attachments")
3. Fields: Individual form fields with types and configurations

Creating Form Templates

Via Admin UI:

• Navigate to /admin → "Deviation Forms" or "CAPA Forms" tab
• Click "Create Template" or "Add Template"
• Build template with sections and fields
• Save and optionally set as default

Via API:

# Deviation form templates
# See backend/app/routers/deviation_form_templates.py

# CAPA form templates
# See backend/app/routers/capa_form_templates.py

# Or use the admin UI:
# Navigate to /admin/deviation-form-templates or /admin/capa-form-templates

Form Creation with Templates

Deviation Forms (/deviations/new): When creating a deviation:

1. Template selector appears at top of form
2. User selects template or creates new one via modal
3. Form template is loaded (default or selected)
4. Sections and fields are rendered dynamically
5. AI suggestions can be enabled per field
6. Form data is stored in DeviationFormData (JSON)
7. Related documents/CAPAs stored via many-to-many associations

CAPA Forms (/capa): When creating a CAPA:

1. Template selector integrated into CAPA creation form
2. User selects template or creates new one via modal
3. Form template is loaded (default or selected)
4. Sections and fields are rendered dynamically
5. Form data is stored in CAPAFormData (JSON)
6. Related documents/deviations stored via many-to-many associations

Field Types Supported

• Basic: text, textarea, select, number, date, boolean
• Pickers: document_picker, capa_picker (deviation forms), deviation_picker (CAPA forms)
• Files: image, file (with upload support)

Implementation Notes

• Templates are versioned (changes only affect new forms)
• Form data stored as JSON for flexibility
• All endpoints require authentication
• Public endpoints (/public/active, /public/default) require valid user session
• Templates can be cloned, activated/deactivated, and set as default

See docs/site/content/user/form-builder.md for comprehensive user documentation.

Adding New Tests

1. Backend Tests

# backend/tests/test_new_feature.py
import pytest
from fastapi.testclient import TestClient
from app.main import app

client = TestClient(app)

def test_new_endpoint():
    response = client.get("/api/new-endpoint")
    assert response.status_code == 200
    assert response.json()["status"] == "success"

2. Frontend Tests

// frontend/__tests__/NewComponent.test.tsx
import { render, screen } from '@testing-library/react';
import { NewComponent } from '../components/NewComponent';

test('renders new component', () => {
  render(<NewComponent />);
  expect(screen.getByText('New Component')).toBeInTheDocument();
});

3. E2E Tests

// frontend/e2e-tests/tests/new-feature.spec.ts
import { test, expect } from '@playwright/test';

test('new feature workflow', async ({ page }) => {
  await page.goto('/new-feature');
  await expect(page.locator('[data-testid="new-feature-title"]')).toBeVisible();
  // ... test steps ...
});

Troubleshooting

Common Issues

Backend Issues

Database Connection Failed

# Check if PostgreSQL is running
docker ps | grep postgres

# Restart database
docker-compose restart postgres

# Check connection
poetry run python -c "from app.database import get_db; print('Connected')"

Import Errors

# Ensure virtual environment is activated
poetry shell

# Reinstall dependencies
poetry install

# Check Python path
poetry run python -c "import sys; print(sys.path)"

Frontend Issues

Build Failures

# Clear node modules
rm -rf node_modules
pnpm install

# Clear Next.js cache
rm -rf .next
pnpm dev

Type Errors

# Regenerate API types
pnpm run gen:api:types

# Check TypeScript configuration
pnpm type-check

Docker Issues

Container Won't Start

# Check Docker status
docker ps -a

# View container logs
docker logs container_name

# Rebuild containers
make dev-live-rebuild

Port Conflicts

# Check port usage
lsof -i :3000
lsof -i :8000

# Stop conflicting services
# ... stop services using ports ...

Getting Help

Documentation Resources

• docs/troubleshooting.md: Comprehensive troubleshooting guide
• docs/api.md: API documentation
• docs/testing.md: Testing guidelines
• AGENTS.md: Detailed development guidelines

Team Resources

• Slack Channel: #innoqualis-dev
• GitHub Issues: Create issues for bugs and feature requests
• Code Reviews: All changes require peer review
• Standup Meetings: Daily sync at 9:00 AM

Best Practices

Code Quality

• Write Tests First: Follow TDD principles
• Use Type Hints: Always type Python functions
• Follow Conventions: Use Black, isort, ESLint, Prettier
• Document Code: Add docstrings and comments
• Keep Functions Small: Single responsibility principle

Git Workflow

• Feature Branches: Create branches for each feature
• Conventional Commits: Use feat:, fix:, docs: prefixes
• Small Commits: Make frequent, focused commits
• Pull Requests: All changes go through PR review
• Clean History: Squash commits before merging

Security

• Never Commit Secrets: Use environment variables
• Validate Input: Sanitize all user inputs
• Use HTTPS: Always use secure connections
• Follow RBAC: Implement proper access controls
• Audit Logs: Log all critical operations

Performance

• Database Queries: Optimize queries, avoid N+1 problems
• Caching: Use Redis for frequently accessed data
• Bundle Size: Keep frontend bundles small
• Lazy Loading: Load components on demand
• Monitoring: Use performance monitoring tools

Next Steps

Learning Resources

1. Read the Documentation: Start with docs/README.md
2. Explore the Codebase: Look at existing implementations
3. Run Tests: Understand the testing patterns
4. Join Team Meetings: Attend daily standups
5. Ask Questions: Don't hesitate to ask for help

First Tasks

1. Set Up Environment: Complete the setup process
2. Run All Tests: Ensure everything works
3. Make Small Changes: Fix a minor bug or add a small feature
4. Submit PR: Go through the full development workflow
5. Get Feedback: Learn from code review comments

Advanced Topics

• AI Integration: Learn about OpenAI API integration
• Compliance: Understand regulatory requirements
• Performance: Optimize system performance
• Security: Implement security best practices
• DevOps: Learn deployment and monitoring

Support

Getting Help

• Documentation: Check docs/ directory first
• Team Chat: Ask in #innoqualis-dev Slack channel
• Code Reviews: Get feedback from team members
• GitHub Issues: Report bugs and request features
• Office Hours: Schedule 1:1 with team leads

Contributing

• Read Contributing Guide: See CONTRIBUTING.md
• Follow Code Style: Use provided linting tools
• Write Tests: Ensure adequate test coverage
• Document Changes: Update relevant documentation
• Be Respectful: Follow code of conduct

---

Welcome to the InnoQualis team! 🎉

This guide should get you started quickly. Remember to ask questions, explore the codebase, and don't hesitate to reach out for help. The team is here to support your success!