CareerForge
Info Portal
Internal reference documentation for the CareerForge career development platform — architecture, user guides, forms workflows, and the complete user directory.
Last updated: 27 April 2026 · Internal use only
Table of Contents
- 01What is CareerForge?
- 02The Four Portals
- 03System Architecture
- 04Backend Services
- 05Authentication & SSO
- 06Data Flow
- 07Employee Portal Manual
- 08Coach Portal Manual
- 09Manager Portal Manual
- 10Admin Portal Manual
- 11Forms & Commitments
- 12AI Mechanics
- 13API Reference
- 14Users & Roles Directory
- 15Access Credentials
- AIAI Deep Dive — Architecture, Guardrails & Roadmap
What is CareerForge?
CareerForge is an internal career development platform for a software IT outsourcing company with eight engineering practices. It connects employees, coaches, and practice managers in a structured, data-driven growth workflow — from skill assessment through structured commitments to verified sign-off.
Key Concepts
- Pseudo-ID — a privacy-preserving identifier derived from Keycloak's
subvia HMAC-SHA256. Format:EMP-XXXXX. PII is encrypted at rest (AES-256-GCM); the pseudo-id is the only identifier visible to managers and coaches without explicit employee consent. - IDP Form — Individual Development Plan created by a coach, committed by the employee, signed off by both coach and manager.
- 1-on-1 Form — Regular meeting summary created by the coach, committed to by the employee.
- Commitment — A formal record of the employee's acceptance of a form. Required before sign-off workflow begins.
- Practice — An engineering discipline (Frontend, Backend, Cloud & DevOps, Data, ML/AI, Mobile, QA, Security). Each practice has one Practice Manager and is served by one or more Coaches.
- Opportunity — A growth opportunity that employees can discover and apply to. Coaches publish skill-development opportunities; managers publish project, bench, L&D, and R&D opportunities.
- Opportunity Match Score — A 0–100% figure computed by the AI Workflow engine showing how well an employee's skills overlap with an opportunity's requirements. Displayed as a colour-coded badge: green ≥70%, yellow 40–69%, grey <40%.
- AI Workflow Config — An admin-managed record that defines which AI provider/model drives all suggestions across all portals. Only one config is active at a time. Each feature (role suggestions, coaching advice, mentor matching, etc.) has its own TTL, guardrails, and optional model override. The active config can point to a cloud provider or a self-hosted LLM (Ollama, LM Studio, vLLM).
- Mentor Matcher — Algorithm in the Coach portal that scores mentors against an employee: 55% skill coverage overlap, 30% seniority fit (gap ≤ 2 ideal), 15% current assignment load. Filters to mentors who are available and at equal or higher seniority.
The Four Portals
CareerForge is delivered as four separate single-page applications, each scoped to one user role, with independent authentication.
System Architecture
CareerForge follows a microservices architecture deployed on Kubernetes. Each portal has a dedicated backend API service; a shared PostgreSQL database holds canonical state; Redis provides caching and session storage.
Backend Services
Each FastAPI service owns a bounded domain. Services share the same PostgreSQL database but access only their own tables.
Employee Service :8002 / employee-service
Core employee operations. Verifies JWTs from the employee Keycloak realm. Decrypts PII only after consent token validation.
Responsibilities
- Profile retrieval (anonymised by default, full PII with consent token)
- Skill heatmap & readiness score aggregation
- Opportunity discovery and application submission
- AI coaching advice generation
- PII reveal request (generates consent token via notification)
- Audit log write for all employee actions
Coach Service :8003 / coach-service
Coach workspace operations. Verifies JWTs from the coach Keycloak realm.
Responsibilities
- Opportunity CRUD (create, list, accept/reject applications)
- Form creation (IDP, 1-on-1, nomination, performance review)
- Coach sign-off on committed forms
- Team analytics (skill heatmap, applications pipeline, cohort readiness)
- Employee profile lookup (anonymised or PII-unlocked)
- Mentor Matcher —
GET /employees/{pseudo_id}/mentor-suggestions?top_n=N— scores available mentors against a given employee using skill overlap (55%), seniority fit (30%), and load (15%); returns ranked list with match %, skill overlap, seniority gap, and active assignment count - Market intelligence and AI coaching advice (per-feature TTL from active AI Workflow Config)
Manager Service :8004 / manager-service
Practice Manager views. Verifies JWTs from the employee Keycloak realm; additionally requires the manager realm role. Runs as a separate image: careerforge-manager-backend:latest.
Responsibilities
- Team member listing (filtered by
line_manager_pseudo_id) - Team member detail view (readiness score, skills, active forms)
- Manager sign-off on forms (after coach sign-off is done)
- Pending sign-off queue
- Team analytics (skill gaps, progression heatmap)
- Opportunity Publishing — Managers create and publish opportunities of type
project,bench,lnd,rnd, orrole; includes publish/unpublish lifecycle - AI Candidate Match —
POST /opportunities/{id}/ai-match— deterministic skill-overlap scoring across all employees + AI narrative for top 3 candidates; results cached per theopportunities_ai_matchfeature TTL from the active AI Workflow Config
Admin Service :8000 / admin-service
Platform administration. Verifies JWTs from the admin Keycloak realm.
Responsibilities
- Skills taxonomy management (create, update, deprecate skills)
- System-wide analytics and KPI dashboards
- Bulk employee data import
- Global audit log browser
- ML model training job triggers
- Keycloak user synchronisation
- AI Workflow Connector — Full CRUD for
AIWorkflowConfigrecords; activate/deactivate; per-feature coverage config (TTL, guardrails, model override per feature per portal); self-hosted LLM support viabase_url; sources catalog endpoint; AI suggestion audit log browser
AI Workflow Config structure
The active config drives every AI call across all portals. Coverage config structure:
{
"employee": {
"role_suggestions": { "enabled": true, "ttl_hours": 24, "guardrails": { "max_roles": 5, "min_match_score": 0.3 }, "model_override": null },
"learning_path": { "enabled": true, "ttl_hours": 48, "guardrails": { "max_nodes": 12, "include_certs": true }, "model_override": null },
"opportunities_matching": { "enabled": true, "ttl_hours": 6, ... },
"ai_chat": { "enabled": true, "ttl_hours": 1, ... }
},
"coach": {
"coaching_advice": { "enabled": true, "ttl_hours": 4, "guardrails": { "require_pseudo_id": true, "safe_mode": true }, ... },
"market_intelligence": { "enabled": true, "ttl_hours": 6, ... },
"mentor_matching": { "enabled": true, "ttl_hours": 24, "guardrails": { "min_match_score": 0.2, "max_mentors": 5 }, ... }
},
"manager": {
"team_readiness": { "enabled": true, "ttl_hours": 24, "guardrails": { "show_names": false }, ... },
"opportunities_ai_match": { "enabled": true, "ttl_hours": 2, "guardrails": { "max_candidates": 10, "min_match_score": 0.3 }, ... }
}
}
Authentication & SSO
All portals use OpenID Connect with PKCE (Proof Key for Code Exchange). No passwords are stored by CareerForge — credentials live in Keycloak.
Keycloak Realms
| Realm | Used by | Issuer URL |
|---|---|---|
employee | Employee Portal, Manager Portal | https://auth-demo.lupulup.com/auth/realms/employee |
coach | Coach Portal | https://auth-demo.lupulup.com/auth/realms/coach |
admin | Admin Portal | https://auth-demo.lupulup.com/auth/realms/admin |
Manager Role Check
The Manager Portal uses the employee realm but additionally requires the manager realm role to be present in the JWT. If the user authenticates successfully but lacks this role, they are redirected to /access-denied.
OIDC PKCE Flow
Stored in sessionStorage. Used for PKCE security.
With response_type=code, code_challenge_method=S256.
Credentials never touch the portal app.
Portal exchanges code + verifier for access_token + refresh_token.
Backend validates signature against Keycloak's JWKS endpoint.
PII Privacy Model
Employee personally identifiable information (full name, email, phone) is stored encrypted with AES-256-GCM. The encryption key is derived per-employee and stored in a secrets manager.
Coaches and managers see only pseudo-ids by default. To reveal PII, the employee must explicitly approve a PII reveal request — a push notification is sent, and on approval a short-lived consent token is issued. The requesting party redeems the token to temporarily decrypt the profile.
Data Flow
Opportunity & Application Flow
Form Sign-off Flow
Manager Opportunity & AI Match Flow
Pseudo-ID Derivation
Each employee's identifier is derived deterministically from their Keycloak sub (UUID) using HMAC-SHA256 + Base32:
HMAC_KEY = bytes.fromhex("77b1c3ed6d3a24931aef148f4099eabc355c79de084f5c10701d5de6044d16f0")
def derive_pseudo_id(keycloak_sub: str) -> str:
digest = hmac.new(key=HMAC_KEY, msg=sub.encode(), digestmod=hashlib.sha256).digest()
encoded = base64.b32encode(digest[:4]).decode("ascii")[:5]
return f"EMP-{encoded}"
This means pseudo-ids are stable (same sub → same id), reversible only by someone with the HMAC key, and safe to expose to coaches and managers.
Employee Portal Manual
The Employee Portal at demo.lupulup.com is your personal career development hub. Sign in with your company SSO credentials (employee realm).
Dashboard
The dashboard shows your current readiness score (0–100), recent activity, skill heatmap, and quick links to pending actions. The readiness score is computed from your skills, certifications, education, learning history, and engagement.
My Profile
- View your current role, seniority, specialisation, and experience years.
- Your Pseudo-ID is displayed — this is how coaches and managers identify you in their portals.
- PII (name, email, contact) is hidden by default. To share it temporarily with a coach or manager, they submit a PII reveal request and you approve it from a push notification.
- Update career goals, learning style preference, and mentoring availability.
Skills & Heatmap
- Browse your skill inventory — each skill has a proficiency level (beginner → expert) and a last-validated date.
- The heatmap shows coverage across technical domains, highlighting gaps relative to your seniority benchmark.
- Earn skill validation by completing courses, certifications, or coach-verified IDP milestones.
Opportunities
- Browse open opportunities posted by coaches and managers — project rotations, bench assignments, certification sponsorships, L&D programmes, R&D initiatives, speaking slots, and mentoring assignments.
- Each opportunity shows a Match Score badge computed from your skill profile vs the opportunity's required skills: ≥70% (strong match) · 40–69% (partial) · <40% (low)
- Filter by specialisation, seniority, type, and tags.
- Apply with a cover note. You can track application status (pending → accepted / rejected) under My Applications.
- Withdraw a pending application at any time.
My Forms
- View all forms addressed to you (IDP, 1-on-1, nomination, performance review).
- A form in pending_commitment status requires your action — read the content and click Commit to accept the goals.
- Once committed, the form moves through coach sign-off → manager sign-off → ACTIVE.
- Active forms are your live commitments. Check milestones and update progress.
AI Advice
- Ask the AI coach for personalised guidance — career path options, skill gap recommendations, learning resource suggestions.
- The AI uses your anonymised profile (no PII). You can provide extra context in free text.
- Save useful advice snippets to your profile notes.
Notifications
- Receive in-app notifications for: new form assignments, application decisions, PII reveal requests, form sign-off completions.
- Approve or deny PII reveal requests directly from the notification panel.
Coach Portal Manual
The Coach Portal at demo-coach.lupulup.com is your workspace for managing employee development. Sign in with your coach realm credentials.
Dashboard
Overview of your active opportunities, pending applications, forms awaiting your sign-off, and cohort readiness trends.
Opportunities
Set title, description, specialisation tags, seniority range, and application deadline. Published immediately — employees can discover and apply.
See each applicant's pseudo-id, readiness score, and cover note. Request PII reveal if you need to contact them directly.
Accepted applicants are notified automatically. Rejected applicants receive a note. You can change decision while the opportunity is open.
Forms
Creating a Form
Select form type, target employee pseudo-id, and optionally assign a practice manager. Form types:
- IDP — Individual Development Plan. Full goals, milestones, timeline, skills targeted.
- One-on-One — Meeting summary. Discussion topics, action items, follow-up date.
- Nomination — Recommend an employee for a role, award, or programme.
- Performance Review — Structured 360° review with ratings and narrative.
After creation, the form status is pending_commitment — waiting for the employee to commit.
Sign-off
Once the employee commits, the form appears in your Sign-off Queue. Review the commitment, add coach notes, and sign. The form then moves to the manager for final sign-off (if a manager is assigned).
Mentor Matcher
On any employee's detail page, the Mentor Match card shows the top-N mentors best suited to that employee — ranked by a weighted score:
- 55% — Skill coverage: fraction of the employee's skills covered by the mentor's skill set.
- 30% — Seniority fit: gap of 0–2 levels scores full marks; each extra level deducts 0.15.
- 15% — Load: mentors with fewer active assignments score higher.
Mentors are only considered if they have available_for_mentoring = true and are at equal or higher seniority than the employee. Match scores are colour-coded: green ≥70%, yellow 40–69%, grey <40%.
AI Coaching Advice
On an employee's detail page, enter a coaching question — the AI receives a strictly-grounded context (the anonymised profile, top-matched opportunities, Industry 4.0 readiness snapshot, and the active company strategy) and returns auditable advice. The system prompt forbids any invented education, certifications, or metrics; Perplexity-style citation markers and raw UUIDs are scrubbed. Every call is logged in the AI suggestion audit trail with a context_used breakdown showing which pillars actually fed the answer. Users can mark each suggestion accepted / dismissed / rated 1–5 via POST /api/v1/ai-suggestions/{log_id}/feedback; admins see acceptance rate and average rating per suggestion_type at /api/v1/admin/ai-workflow/audit/feedback-summary. The TTL and guardrails for coaching advice are configurable per-feature in the Admin → AI Workflow Connector.
Analytics
- Skill Heatmap — coverage and gaps across your coaching cohort.
- Applications Pipeline — funnel view from applied to placed.
- Cohort Readiness — average and distribution of readiness scores.
- Opportunity Performance — fill rate, time-to-fill, application volume.
Manager Portal Manual
The Manager Portal at demo-manager.lupulup.com gives Practice Managers visibility into their team's career progression. Requires the manager Keycloak role.
manager realm role in the employee Keycloak realm can access this portal. If you see an "Access Denied" page, contact IT to have the role assigned to your account.
Dashboard
Summary of your team: total members, average readiness score, pending sign-offs, active IDPs, and recent form activity.
My Team
- List of all employees where your pseudo-id is their
line_manager_pseudo_id. - Each row shows pseudo-id, seniority, specialisation, readiness score, and active form count.
- Click a team member to see their full profile: skill breakdown, active forms, readiness score components, and progression history.
- Team member PII is hidden by default. Submit a reveal request to temporarily view name and contact.
Sign-offs
- Forms appear in your queue after the employee has committed and the coach has signed off.
- Review the full form content, coach notes, and employee commitment.
- Add your manager comments and sign off to mark the form ACTIVE.
- You can request clarification from the coach before signing.
Opportunities
Managers can create and publish opportunities visible to all employees in the Employee Portal.
Set title, description, required skills, type (project, bench, lnd, rnd, role), and seniority range. New opportunities start as draft — not yet visible to employees.
Click Publish to make the opportunity live in the Employee Portal. Use Unpublish to pull it back without deleting it.
Click AI Match on any opportunity. The system scores all employees by skill overlap and returns a ranked candidate list with match percentages, matched skills (green), and missing skills (strikethrough). The top 3 candidates also receive an AI-generated narrative. Results are cached per the opportunities_ai_match TTL in the active AI Workflow Config.
Practice Managers
| Manager | Pseudo-ID | Practice |
|---|---|---|
| alice.johnson | EMP-HCUZE | Frontend |
| bob.smith | EMP-OJ5RQ | Backend |
| chris.evans | EMP-IRUKS | Cloud & DevOps |
| diana.morgan | EMP-EN5FU | Data Engineering |
| edward.hughes | EMP-HZ2CC | ML / AI |
| fiona.taylor | EMP-VGB6P | Mobile |
| grace.chen | EMP-LNASW | Quality Assurance |
| henry.kim | EMP-LCJZV | Security |
Admin Portal Manual
The Admin Portal at demo-admin.lupulup.com is for platform operations. Restricted to users in the admin Keycloak realm.
Skills Taxonomy
- Browse the global skill catalogue — ~200 skills across all engineering practices.
- Add new skills: set name, category, description, and related skills.
- Deprecate obsolete skills — existing proficiency records are preserved but the skill is hidden from new entries.
- Merge duplicate skills: all employee records are remapped.
System Analytics
- Platform-wide KPIs: active users (7d / 30d), forms created, form completion rate, opportunity fill rate.
- Drill down by practice, seniority, or coach.
- Export data as CSV for external reporting.
Audit Log
- Every create / update / delete action is logged with actor pseudo-id, action type, resource, and timestamp.
- Filter by date range, actor, resource type, or action.
- Audit log is append-only — records cannot be modified or deleted through the UI.
ML Model Management
- View current model versions tracked in MLflow.
- Trigger re-training jobs (runs as a Kubernetes Job).
- Promote a model version to production — employee service will use it for readiness scoring.
AI Workflow Connector
Configure which AI engine drives all CareerForge suggestions. Navigate to Connectors → AI Workflow tab.
- Create a config — choose provider (
anthropic,openai,perplexity,gemini,self_hosted), model name, and API key. For self-hosted, set theBase URL(e.g.http://ollama:11434) — the connector auto-appends/v1if needed. - Activate — only one config is active at a time. Activating one automatically deactivates all others. The active config is cached for 5 minutes across all services.
- Per-feature coverage — expand each feature panel (e.g. role_suggestions, coaching_advice, mentor_matching, opportunities_ai_match) to set:
- Enabled toggle
- TTL (hours) — how long a suggestion result is cached before re-generation
- Model override — use a different model for this feature only
- Guardrails — key/value pairs (e.g.
max_roles: 5,safe_mode: true)
- Suggestion Audit tab — browse the append-only log of every AI call: suggestion type, portal, feature, employee pseudo-id (if applicable), model used, sources used, generated_at, expires_at.
Forms & Commitments
Forms are the core unit of documented career development in CareerForge. Every form goes through a structured lifecycle from creation to full sign-off.
Form Lifecycle
Form Types
| Type | Creator | Manager Sign-off | Typical Content |
|---|---|---|---|
idp | Coach | Required | Goals, milestones, target skills, timeline, success criteria |
one_on_one | Coach | Optional | Meeting agenda, discussion points, action items, next meeting date |
nomination | Coach | Required | Role/programme being nominated for, justification, evidence |
performance_review | Coach | Required | Competency ratings, strengths, development areas, overall rating |
Commitment Object
When an employee commits to a form, a FormCommitment record is created:
{
"form_id": "uuid",
"employee_pseudo_id": "EMP-XXXXX",
"status": "committed", // or: in_progress, completed, abandoned
"commitment_date": "2026-04-21T10:00:00Z",
"employee_notes": "I accept and will focus on Kubernetes certification first.",
"progress_percentage": 0,
"milestones_completed": 0
}
AI Mechanics
How each AI suggestion is computed across all four portals — what is deterministic versus LLM, the per-feature guardrails it respects, and the audit log entry it produces. Click any row to expand.
Role SuggestionsLLM + RAG
Employee’s skills, seniority, and specialisation are serialised into a prompt. The active AI Workflow provider (Claude / Perplexity / OpenAI / Gemini / self-hosted) is called via get_role_suggestions_for_profile().
RAG searches ai_documents for relevant roles-catalogue and trend chunks, prepends them as context, then asks the LLM to return the top N most relevant roles for this profile.
A deterministic match_score (0.0–1.0) is appended per role: |emp_skills ∩ required| / |required|. Roles below min_match_score are filtered before returning.
Learning PathLLM + RAG
Skill set, career goal, and seniority gap are combined with RAG context (skill docs, cert data). The LLM is asked to return a node-graph JSON with node types: skill, cert, project, milestone.
Output is validated for JSON structure. Nodes exceeding max_nodes are trimmed (lowest-priority last). If include_certs: false, cert nodes are stripped. Cached 48h — long enough to avoid redundant calls, short enough to reflect market shifts.
Opportunity Match ScoreDeterministic
No LLM. For every published opportunity, at query time:
Badge colour: green ≥70%, yellow 40–69%, grey <40%. Score attached to every row in the opportunities list — no separate call needed.
Opportunities below min_match_score are still shown but de-emphasised. Disabling the feature hides all badges.
No row written. The score is computed inline per opportunity in the same SQL query that returns the list — it is not a discrete suggestion. The coach’s “Suggest Employees for Opportunity” uses the same scoring formula and does write an audit row (see Coach Portal → Suggest Employees).
Career Advice (Get Advice)LLM + 4 pillars
Single-shot question on the dashboard’s Get Advice card. Backend assembles a strict 4-pillar context: anonymised profile (skills resolved from SkillTaxonomy — UUIDs never sent), top 5 active opportunities ranked by skill overlap, the latest Industry 4.0 readiness snapshot, and the active workflow’s company strategy notes.
Profile too thin (< 3 skills, no specialisation/seniority/career goal) → 422 refusal rather than fabrication. Off-context questions (e.g. “where did I go to university?”) get a clean “not in your profile, please add it” refusal.
Output passes a regex scrub for [1][2] / [digest] citation markers and stray UUIDs. Rendered as Markdown via the shared CfMdText component (marked + DOMPurify).
AI Career Advisor (multi-turn chat)LLM + 4 pillars
The conversational Career Advisor in My Profile. Same 4-pillar grounding as Career Advice, but with the last 6 turns of conversation history prepended. Client owns session state — backend is stateless. History is capped at 10 turns; user message at 1500 chars.
Same anti-fabrication system prompt; same UUID / citation scrubber on the way out.
AI Career SummaryLLM
One-click generation of a first-person career narrative on the employee’s profile page. Skill UUIDs resolved to taxonomy names before the prompt; education_json is never sent (the wizard does not collect it — any value would be seed-only and would surface as fabrication).
Strict system prompt: use only the supplied profile fields; never invent education, certifications, employers, projects, salaries, locations, or numeric metrics. Output split on the literal "Growth trajectory:" token into two displayed blocks.
422 refusal when the profile is too thin. Citation markers and UUIDs scrubbed post-LLM.
Skill Gap (target role readiness)LLM + RAG
Target role title + the employee’s current skill set. RAG retrieves role-requirement docs and recent job-posting data. The model returns the top 8 skills the employee is missing or weak in, with required_level and market_demand grounded in connector data.
Used both by the employee’s Learning page (current vs target role) and surfaced on the dashboard’s readiness card.
Output truncated to max_skills, sorted by market_demand descending. Only absent or below-required-level skills are included.
Coaching AdviceLLM + 4 pillars
Coach submits a question referencing an employee by pseudo-id. Backend assembles the same 4-pillar context as the employee Career Advice: anonymised profile (skills resolved from SkillTaxonomy), top-matched opportunities, latest readiness snapshot, active company strategy.
require_pseudo_id: true means the endpoint rejects any request without a valid, existing pseudo-id — no anonymous bulk queries. Strict system prompt forbids invented education, certifications, employer history, projects, salaries, locations, metrics; bans citation markers and raw UUIDs.
Rendered as Markdown via CfMdText. Post-LLM regex scrub strips [1][2] / [digest] / UUID artefacts.
Market TrendsLLM + RAG
Connectors (tech news feeds, job market APIs) ingest data hourly via Celery Beat into ai_documents. The RAG layer retrieves the freshest chunks. The LLM synthesises them into trend objects: { topic, trend_direction, summary, confidence, sources[] }.
Not employee-specific — one shared cache entry per topic. Individual source URLs are cited so coaches can verify any claim.
Emerging Roles (catalogue)LLM + RAG
Coach Catalogues page surfaces emerging IT roles per specialisation (or general). RAG retrieves connector docs scoped to the spec; the LLM returns a JSON array of { role_title, demand_level, avg_salary_range, required_skills[], growth_rate_percent }.
Cache is keyed by specialisation, so all coaches looking at the same practice share one warm entry. Distinct from Market Trends: roles & demand levels, not directional signals.
Output is parsed as JSON; _parse_json() strips fences and falls back to empty list (never free text into the UI).
Suggest Employees for OpportunityDeterministic
Coach asks “who in the org should I shortlist for this opportunity?”. No LLM. Pool: all employees meeting the opportunity’s minimum seniority. Score per candidate:
Top-N returned (default 5), with skill overlap and mentoring-availability flag.
Zero LLM cost. Tagged model_used: "rule_based" so the audit page can distinguish it from LLM rankings.
Mentor Match (employee & opportunity)Deterministic
Two sibling endpoints share one scoring formula:
GET /employees/{pseudo}/mentor-suggestions— on the employee detail pageGET /opportunities/{id}/mentor-suggestions— on the opportunity detail page
Pool: employees with available_for_mentoring=true, seniority ≥ target’s, pseudo_id ≠ target. Weighted score per candidate:
Candidates below min_match_score excluded. Capped at max_mentors. Zero LLM cost — entirely data-model driven. Only pseudo-ids exposed — no names. Tagged model_used: "rule_based" in the audit log.
Opportunity AI Candidate MatchLLM + Deterministic
Phase 1 — Deterministic (all 212 employees):
Sorted by score desc. Filtered by min_match_score. Capped at max_candidates.
Phase 2 — LLM narrative (top 3 only): opportunity description + top-3 anonymised profiles sent to the active provider for a concise hiring narrative per candidate.
Short 2h TTL — opportunity requirements may change. LLM only sees: pseudo_id, specialisation, seniority, skill_list. No names, no email, no personal data.
Team Readiness ScoreML Model
Not an LLM — an XGBoost model trained on historical skill progression, certification completions, IDP activity, and engagement data. Produces a score 0–100 per employee.
Features: skill_count, cert_count, idp_completion_rate, seniority_index, last_activity_days, engagement_score. Model version tracked in MLflow; promoted to production by an admin trigger — no code deploy required.
show_names: false enforces pseudo-id display at all times. PII reveal requires an explicit employee consent flow. 24h TTL matches the daily reporting cadence.
Outcome Feedback LoopDid it work?
The audit log answers “what did the AI say?”. The new ai_suggestion_feedback table answers “did it work?”. Each suggestion log row can be paired with one or more feedback events:
- accepted — user actioned the suggestion (applied to a role, enrolled in a learning node, asked a follow-up)
- dismissed — user explicitly rejected it
- rated — thumbs / 1–5 stars; carries a numeric rating
- auto — system-observed outcome (readiness delta 30d after a learning path was generated, application result, etc.)
Joining the two over time gives the platform real signal: acceptance rate per suggestion_type, average rating, correlation with downstream readiness changes — the data needed to tune prompts on evidence rather than vibes.
Authorisation on POST: the suggestion must either belong to the caller (matching employee_pseudo_id) or be portal-wide / unscoped. Bad action / out-of-range rating → 422.
AI Observability — Prometheus metricsGrafana-ready
The HTTP-level instrumentor already records request latency and status. These metrics target the AI surface — the things you can’t infer from request logs:
- cf_llm_calls_total{provider, model, outcome} — success / empty / http_error / exception
- cf_llm_call_duration_seconds{provider, model} — histogram, 0.5..90s buckets
- cf_suggestion_calls_total{suggestion_type, portal, outcome} — success / empty_answer / refused_thin_profile / feedback_accepted / feedback_dismissed / feedback_rated
- cf_llm_artifacts_scrubbed_total{kind} — citation_numeric / citation_named / uuid
- cf_ai_cache_lookups_total{cache_kind, outcome} — hit / miss / expired
A regression like the Perplexity citation leak we caught manually mid-test would now alert itself: cf_llm_artifacts_scrubbed_total{kind="citation_numeric"} stops being zero. Same for silent regressions:
- P95 latency for one provider creeping past 30s → switch model_override
refused_thin_profileratio rising → onboarding wizard issueempty_answerspikes on one suggestion_type → bad prompt template change- Cache hit rate dropping → TTL too aggressive or fingerprint collision
Cost & Abuse ControlsThree layers
- Per-employee daily quota — default 50 LLM-backed calls per
pseudo_idper UTC day./ai-adviceand/ai-chatconsume one slot each. Exhausted → HTTP 429 with a clear message; resets at 00:00 UTC. Redis-backed (INCR + EXPIRE); fail-open if Redis is down. - Per-provider circuit breaker — after 5 consecutive failures (HTTP error / exception) the breaker opens for 120s. Open-state short-circuits the LLM call and returns
""; caller serves the configured fallback. Auto-closes after the cooldown. - Monthly token cost cap — rough estimate (chars/4) recorded per provider in
cf:cost:YYYY-MM.coverage_config.__cost__.monthly_token_capon the active workflow sets the limit; once hit, every LLM call returns""until the next month. Unset = unlimited.
All three are fail-open: if Redis or a counter is unreadable, the call proceeds rather than blocking the portal.
Audit Evidence Trail — verbatim prompt & answerCompliance-ready
Every conversational suggestion (coaching_advice, ai_chat) records into payload_json:
- The system prompt verbatim (the strict-grounding rules + role declaration)
- The full user prompt — all four PROFILE / OPPORTUNITIES / READINESS / STRATEGY blocks exactly as the model saw them
- The full answer after post-LLM scrub
- The context_used breakdown (skills count, opportunities ranked, readiness/strategy presence)
- Every recorded
ai_suggestion_feedbackevent (accepted / dismissed / rated 1–5)
This is the artefact HR / compliance / legal asks for when reviewing why the AI made a recommendation. No PII is in any of these blocks — only pseudo-IDs and aggregated profile fields, by construction.
Browser saves directly. Print-to-PDF for tickets / audits.
Three audiences, one artefact:
- Compliance / GDPR Article 22 — demonstrates that any AI-assisted decision had a documented basis (the four pillars) and that the user retained the ability to contest / accept / dismiss.
- Coach / HR review — when an employee disputes advice, the conversation is fully reproducible.
- Prompt engineering — reading the actual prompt the model saw is the fastest way to debug a regression.
Runtime FE Configuration — one bundle deploys everywhereDevOps
The employee portal reads its environment-specific values (Keycloak URL, realm, client-id, API base URL) from window.__APP_CONFIG__ at runtime, populated by a /config.js file rendered from container env at startup.
Build-time VITE_* values still work as a fallback so pnpm dev and existing built bundles keep working unchanged.
Same image promotes from demo → staging → production. Set CF_KEYCLOAK_URL, CF_EMPLOYEE_API_URL, … on the deployment and they take effect on next pod restart.
All four portals (employee, coach, manager, admin) ship the same pattern — each has its own runtime-config.ts + docker-entrypoint.sh + nginx /config.js no-cache block. The cf-portal Helm chart accepts an optional env: array, so a single image promotes from demo → staging → production by overriding CF_KEYCLOAK_URL / CF_<APP>_API_URL at deploy time.
Audit Trail — every suggestion, every portalAppend-only
Every call to any AI function invokes _log_suggestion(portal=, feature=) before returning. The TTL is resolved by _get_feature_ttl(workflow, portal, feature): reads coverage_config[portal][feature].ttl_hours from the active config, falls back to suggestion_ttl_hours global, then to 24h hardcoded default.
The log is append-only — no UPDATE or DELETE is ever issued. Admins browse it via Admin → Connectors → Suggestion Audit tab, filterable by portal, feature, pseudo-id, or date range.
Switching provider in Admin UI takes effect within 5 minutes across all services — no restart required.
Per-feature Config — live guardrail changes, no deployAdmin UI
In Admin → Connectors → AI Workflow, each config has expandable per-feature panels. Per feature you can change:
- Enabled — toggle off without a code change
- TTL hours — how long results are cached before re-generation
- Model override — cheaper model for bulk scoring, smarter model for coaching advice
- Guardrail key/values — any key shown above; live after the 5-min cache window
Set provider to self_hosted. Enter a base_url (e.g. http://ollama:11434). The connector auto-appends /v1 and calls /v1/chat/completions (OpenAI-compatible). Bearer auth optional. 120s timeout.
Works with Ollama, LM Studio, vLLM, or any OpenAI-API-compatible server.
Five-layer cost & resilience stackall providers
Sits in front of every conversational AI endpoint. Layers 1–4 ship benefits regardless of which provider the active workflow uses; layer 5 sits behind a capability predicate that activates only when Anthropic is selected.
- Prompt-shape budget — top-3 opportunities, strategy 800 chars, chat history 4 turns. 25–35% input-token cut on every provider.
- Response-hash cache — Redis key
cf:airesp:<sha>, 5-min TTL. Double-clicks & reloads cost zero. - Multi-replica circuit breaker — Redis-backed open state across all replicas.
- Connector freshness gauge —
cf_connector_last_sync_seconds+ alert at 24h. - Anthropic prompt cache — ephemeral
cache_controlon the system prompt; ~90% input-cost cut on cache hits.
/ai-chat additionally folds a hash of recent turns in so a follow-up question with the same words can’t collide with the original.
Fail-open everywhere: cache, breaker, and quota helpers all degrade to no-op when Redis is unreachable.
Vector cosine RAG retrievalall providers
Every AI function that needs grounding chunks from ai_documents calls _cached_text_search(). That helper now embeds the query via the configured embedding provider (Ollama nomic-embed-text by default, OpenAI text-embedding-3-large when OPENAI_API_KEY is set) and orders ai_document_vectors by cosine distance.
Cosine similarity matches synonyms and paraphrases that LIKE missed: "AWS" ↔ "Amazon Web Services", "k8s" ↔ "Kubernetes", "microservices" ↔ "service-oriented architecture".
If embedding generation fails (no embedding service reachable, transient HTTP error) or the vector query returns nothing, falls through to the original LIKE-based search. Both paths return the same dict shape, so every existing caller is unchanged. RAG is never blocked.
Daily AI-quota indicatorUX
The employee portal sidebar footer shows AI: 48 / 50 today under the user name, polled every 60s. Turns orange when remaining ≤ 5 so the user gets a visible heads-up before hitting the daily ceiling. Hidden when Redis is unreachable (remaining = -1) so the chip never lies.
Reads from GET /api/v1/quota/me which returns the same Redis counter the Phase 1 daily-quota breaker enforces — single source of truth, no drift between what the user sees and what the platform actually applies.
Empty remaining = -1 when the Redis sidecar is down — same fail-open posture as the rest of Phase 1.
For now, employee portal only (apps/employee/src/layouts/AppLayout.vue). Coach / manager / admin portals don’t have a quota concept on their AI surfaces yet, so no chip there. The endpoint itself is auth-scoped to the calling Keycloak subject so each employee only sees their own counter.
Drift gate (two-gate setup)CI
Two complementary CI gates, paired so neither bug class slips through:
alembic check— fails when a SQLModel field has no matching migration column. The bug class that manifests asUndefinedColumn500s after a fresh deploy.pytest test_raw_sql_ghost_tables.py— fails when a router uses raw SQL against a table that has no SQLModel at all. The bug class behind theapi_keysincident.
migrations/env.py ships an _include_object filter that skips index, FK, and unique-constraint naming divergence (cosmetic only — same b-tree, same referential semantics). Real drift — missing columns, nullable changes, missing tables — is still caught.
api_keys table: admin /api-keys returned 500 because the table referenced via raw SQL had no SQLModel and no migration. Caught by the ghost-table test.
coach_forms + skill_taxonomy: SQLModel had fields (employee_committed_at, cc_signed_off_at, manager_signed_off_at, opportunity_id, description) the migrations never created. Caught by alembic check when the strict gate was restored; reconciled via migrations 20260429_0002 + _0003 (both idempotent).
API Reference
All services expose OpenAPI docs at /docs (Swagger UI) and /redoc (ReDoc). Base paths are /api/v1.
Employee Service — demo.lupulup.com/api/v1
Coach Service — demo-coach.lupulup.com/api/v1
Manager Service — demo-manager.lupulup.com/api/v1
Admin Service — demo-admin.lupulup.com/api/v1/admin
Users & Roles
Dev environment user directory. The employee realm contains 108 users (employees + managers); the coach realm contains 2 coaches; the admin realm contains 1 administrator. All dev accounts use the password Demo2026!.
Administrators (admin realm)
| Username | Full Name | Portal | Password |
|---|---|---|---|
admin1 | Admin User | demo-admin.lupulup.com | Demo2026! |
Coaches (coach realm)
| Username | Full Name | Portal | Password |
|---|---|---|---|
coach1 | Andrei Popa | demo-coach.lupulup.com | Demo2026! |
coach.demo | Coach Demo | demo-coach.lupulup.com | Demo2026! |
Employees (employee realm)
| Username | Full Name | Pseudo-ID | Job Title | Seniority | Specialization | Reports To | Keycloak Role |
|---|
Access Credentials
Simulation Accounts (all portals)
| Realm | Users | Password |
|---|---|---|
| employee | 108 accounts — see Users & Roles table above. Notable: employee1, employee2, manager1, alex.manager, ioana.lead, stefan.head | Demo2026! |
| coach | coach1 (Andrei Popa), coach.demo (Coach Demo) | Demo2026! |
| admin | admin1 (Admin User) | Demo2026! |
Container Registry
| Registry | git.lupulup.com |
|---|---|
| Namespace | admin/ |
| Images | careerforge-backend:latest, careerforge-employee:latest, careerforge-coach:latest, careerforge-manager:latest, careerforge-admin:latest |
Kubernetes
| Cluster | k3d-cf-demo (local dev) |
|---|---|
| Namespace | cf-demo |
| ArgoCD | https://argocd.lupulup.com |