Feature #61
closedFeature #21: Enhance Matching Results and AI Assistant with Company Enrichment and Market Insights
[MatchingAgent] Market-insights aggregate endpoint (skills/location/work-mode demand)
0%
Description
**Surface:** MatchingAgent (`MatchingAgent/`, FastAPI, `src/api/`).
## Description
Add a FastAPI endpoint that computes market-insights aggregates over the active-offer corpus, scoped by role and optional filters (location, seniority):
- Top in-demand skills with offer counts (e.g. "Java (8,974 offers), Kubernetes (7,086)").
- Location trends (which locations have the most offers for the role).
- Work-mode distribution (e.g. "80% hybrid").
Requirements:
- New endpoint (e.g. `POST /v1/market-insights`) returning a typed JSON aggregate.
- Aggregation must be algorithmic (no LLM call), reusing the active-offer definition (isActive + extractedAt window + not expired) consistent with the rest of the corpus.
- Performance: respond within budget when serving a 100+ matching-result batch (cache/precompute as needed).
- pytest unit coverage for the aggregation logic (skills counts, location ranking, work-mode percentages).
Blocks #3 (GetYourJob proxy route).
## Acceptance criteria
- [ ] `POST /v1/market-insights` returns top skills with offer counts, location trends, and work-mode distribution for a given role/filter set.
- [ ] Aggregates use the active-offer definition and contain no LLM calls.
- [ ] Endpoint meets performance budget for a 100+ result batch (documented timing).
- [ ] pytest covers skill counts, location ranking, and work-mode percentages.
**Estimate:** M
RA Updated by Redmine Admin about 2 months ago
- Status changed from Submitted to In development
Claiming for backend dev (MatchingAgent market-insights aggregate endpoint). Moving to In development.
RA Updated by Redmine Admin about 2 months ago
⏸️ Stopping before build — data-home finding: MatchingAgent does NOT own the active-offer corpus, so this aggregate endpoint should be re-homed to GetYourJob (folding #61 into a GYJ-side endpoint, alongside #63). No code written; no fake data.
Evidence:
- MatchingAgent persists no job_offers. Its DB (migrations 001–006 / models.py) holds only users, api_keys, usage_records, subscriptions, rate_limits, matching_sessions. The API is request-driven: callers POST offers (max 5000) into /v1/match-offers & /v1/match-candidates; offers are only kept transiently in a per-user matching_sessions row (7-day TTL, one request's batch). There is no whole-corpus store to aggregate.
- The ONLY corpus access is an OFFLINE batch script (scripts/sync_faiss_index.py, compare_faiss_quality.py) that reaches straight INTO the GetYourJob Postgres via GETYOURJOB_DB_URL and SELECTs FROM job_offers to rebuild the FAISS vector index. That's a CLI/cron tool and a cross-DB reach, not a runtime API path.
- The canonical active-offer rule (isActive + extractedAt window + not expired) is OWNED and computed on the GetYourJob side. The sync script even comments it must "mirror the GetYourJob runtime window (21d + grace)" and only approximates it (createdAt >= NOW() - 30 days) — building the aggregate here would fork/drift this definition.
- FAISS holds vectors for similarity search, not a counts/group-by store, and is a stale offline snapshot — not a sound source for "Java: N offers" market counts.
Recommendation: re-home the market-insights aggregation to a GetYourJob-side endpoint where the active-offer corpus and its canonical filter live (Prisma buildWhereClause / findAll over job_offers). That naturally folds #61 + #63 into one GYJ endpoint. MatchingAgent has no sound, in-boundary path to produce real market-wide counts, and the no-fake-data rule forbids inventing them.
Leaving #61 In development for orchestrator re-route. No branch/PR created.
RA Updated by Redmine Admin about 2 months ago
⚠️ RE-HOMED vers GetYourJob. Investigation : MatchingAgent ne possède PAS de corpus d'offres interrogeable (offres = GYJ Postgres `job_offers`, MA ne garde que des sessions transitoires par requête). Construire l'agrégat ici forkerait la définition canonique « active-offer » et violerait la règle no-fake-data. Décision : l'endpoint market-insights sera construit côté GetYourJob (agrégation sur `job_offers` via le filtre canonique), **fusionnant #61 + #63 en un seul endpoint GYJ**. Reste In development ; livraison côté GYJ (branche feat/redmine-61-market-insights-gyj).
RA Updated by Redmine Admin about 2 months ago
- branch set to feat/redmine-21-insights-ui
- pr_url set to https://github.com/omdev-tech/GetYourJob/pull/239
🏁 done dev — ready for QA (reste In development).
Endpoint market-insights livré côté GetYourJob (re-home #61+#63 fusionnés) : `GET /api/market-insights?role=&location=` (auth getServerSession).
- Agrégat sur le corpus ACTIF via `PrismaJobOfferRepository.findActiveMarketInsightRows` qui RÉUTILISE `buildWhereClause` (isActive + fenêtre extractedAt + non expiré ; roles→roleType, localisations→locationTags) — aucune dérive de la définition active-offer canonique.
- Logique d'agrégation pure et testée : `MarketInsightsService.computeMarketInsights` (top compétences avec comptes réels, top localisations, distribution remote/hybrid/onsite %). Corpus vide → agrégats vides, jamais de chiffre inventé.
- Cache Redis + fallback mémoire, clé par role/location (absorbe le batch 100+).
Tests : 11 (agrégation) + 8 (route : garde auth 401, forwarding filtres, clé cache, 500). vitest vert, build OK (`/api/market-insights` = route ƒ dynamique), tsc propre sur les fichiers touchés.
PR : https://github.com/omdev-tech/GetYourJob/pull/239 (base `dev`). CI Jenkins (branch + pr-head) verte.
RA Updated by Redmine Admin about 2 months ago
RA Updated by Redmine Admin about 2 months ago
- Status changed from Preprod to Shipped
Livré en production avec l'épopée #21 (endpoint market-insights GYJ). Preprod → Shipped.