Enhancement of most of blocking stuff#30
Open
crackmou wants to merge 15 commits into
Open
Conversation
Rework InvoiceParser to insert invoices and update them on conflict (INSERT ... ON CONFLICT (name) DO UPDATE), so records missing from the database are no longer silently skipped. - Replace raw string-interpolated SQL with parameterized queries to fix SQL injection - Parse JSON via json_decode instead of fragile line-by-line parsing - Persist the currency field, which was previously ignored - Split parse() into parseJson/parseCsv/upsert for readability - Add a unique constraint on invoice.name (entity + migration) required by the ON CONFLICT clause Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Untrack the IDE config files under .idea and rely on the existing .idea/* gitignore rule so editor-specific changes are no longer staged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…iton on DB you need to make
…nal, partner) - Add App\Enum\Currency (EUR, USD, GBP, JPY) sourced from data/*; parsing converts via Currency::from() and throws on an unknown currency - Type Invoice::$currency with the enum (Doctrine enumType, column unchanged) - Rework InvoiceRepository::upsert: insert id_external, resolve the partner by name (throws if missing), conflict on the (id_external, partner_id) key - InvoiceParser: forward id_externe + partenaire, fix CSV column indices - Add Makefile `parse` target for `docker-compose run --rm app php bin/console app:parse` Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ie le parsing Outillage CI / qualité de code : - ajoute friendsofphp/php-cs-fixer (^3.75) et phpstan/phpstan (^2.2) en dev - ajoute la configuration .php-cs-fixer.dist.php (règles @symfony) - ajoute le hook ci/git/pre-push-hook (php-cs-fixer + phpstan niveau 3) - ignore les fichiers générés par php-cs-fixer dans .gitignore - documente le hook de pre-push dans le README Parsing des factures, gestion d'erreurs et robustesse : - ParseInvoicesCommand : itère sur la liste des fichiers et capture les erreurs par fichier via SymfonyStyle, au lieu d'enchaîner des appels secs - InvoiceParser : détection du format par extension (pathinfo + match) au lieu de str_contains, lève une exception sur fichier introuvable ou extension non supportée - InvoiceParser : factorise la construction des lignes via getElement() - applique les règles de style @symfony (Yoda conditions, espacements) Données de test : - ajoute des jeux de données pour les cas d'erreur : invoices.xml, invoices_invalid.json, invoices_missing_column.json
…pendances et couvre les nouveaux comportements Architecture - Pattern Strategy pour la lecture (InvoiceReaderInterface + Csv/Json readers, sélection via InvoiceReaderRegistry) : ajouter un format = une classe (OCP). - Value Object immuable InvoiceInput à la place des tableaux à clés magiques (mapping + validation centralisés). - InvoiceParser réduit à de l'orchestration ; extraction d'InvoiceWriterInterface pour dépendre d'une abstraction d'écriture, plus du repository concret (DIP). - Exceptions métier dédiées au lieu de \Exception génériques. Tests / CI - Remplace l'ancien test cassé ; couvre readers, registre, DTO, orchestration et bornes de batch. - Test d'intégration du repository sur PostgreSQL (ON CONFLICT, séquence, jointure partenaire), isolé par transaction/rollback. - Couverture 50 % -> 78,79 % ; PCOV + cibles make coverage et gate dans le hook pre-push (seuil 75 %).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Résumé
Refonte du parser de factures : insertion en lot + streaming, sécurisation des requêtes SQL, et passage à un modèle de données avec partenaires et devises typées.
Changements principaux
🔧 Parser (
src/Service/InvoiceParser.php)preg_split/explode) parjson_decodeetfgetcsven streaming.FLUSH_SIZE = 1000lignes → empreinte mémoire constante, même sur 1M de lignes.UPDATE ... WHERE name = '...'construits par concaténation (faille d'injection SQL) au profit du repository.🗃️ Insertion en lot (
src/Repository/InvoiceRepository.php)upsertBatch(): un seulINSERT ... VALUES (...),(...) ON CONFLICT (id_external, partner_id) DO UPDATE, découpé par paquets de 1000 (limite de paramètres PostgreSQL), paramètres liés.resolvePartnerIds(): résolution des partenaires en une requête, erreur explicite si partenaire inconnu.🧱 Modèle de données
Partner(+PartnerRepository).Currency(EUR/USD/GBP/JPY) —Invoice.currencyn'est plus unstring.Invoice: ajout deidExternalet relationManyToOneversPartner; clé métier unique =(id_external, partner_id).📜 Migrations
…120000invoice.name(étape intermédiaire)…140000partner+ colonnesid_external/partner_id(FK + index)…150000…160000id_external…170000(id_external, partner_id)📦 Données & outillage
data/invoices.csv/invoices.json: ajout des colonnesid_externeetpartenaire.data/invoices_1000k.csv(1 000 000 lignes) pour les tests de charge.Dockerfile:memory_limit = 1024M.Makefile(cibleshelp,parse)..gitignore: exclusion de.idea/.Tests
tests/InvoiceParserTest.phpadapté au nouveau flux.