From 6cb132b0d1b1c1a0d5b1b24b99996107a5df3ecf Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Mon, 18 May 2026 08:52:07 +0000 Subject: [PATCH 1/8] Add invoice scan to transaction design plan Co-authored-by: Gerben Jongerius --- README.md | 1 + docs/INVOICE_SCAN_TRANSACTION_PLAN.md | 258 ++++++++++++++++++++++++++ 2 files changed, 259 insertions(+) create mode 100644 docs/INVOICE_SCAN_TRANSACTION_PLAN.md diff --git a/README.md b/README.md index ad2c2c0..967ea6a 100644 --- a/README.md +++ b/README.md @@ -110,6 +110,7 @@ app/src/main/java/com/pledgerio/app/ - [Account types](docs/ACCOUNTS.md) — Owned vs counterparty accounts, type codes, transaction mapping - [Budgets](docs/BUDGETS.md) — Initial setup, expense groups, API mapping - [Transaction form redesign](docs/TRANSACTION_FORM_REDESIGN.md) — Planned UX for creating transactions (type-first flow, amount hero, contextual account labels) +- [Invoice scan to transaction plan](docs/INVOICE_SCAN_TRANSACTION_PLAN.md) — Proposed OCR + text extraction pipeline to prefill new transactions - [Usability modes](docs/USABILITY_MODES.md) — Guided mode for novices and Power mode for advanced users - [Localization](docs/LOCALIZATION.md) — English, Dutch, and German (extensible) - [Architecture Decision Records](docs/adr/README.md) — Rationale for major technical choices diff --git a/docs/INVOICE_SCAN_TRANSACTION_PLAN.md b/docs/INVOICE_SCAN_TRANSACTION_PLAN.md new file mode 100644 index 0000000..f3e81ae --- /dev/null +++ b/docs/INVOICE_SCAN_TRANSACTION_PLAN.md @@ -0,0 +1,258 @@ +# Invoice and bill scan to transaction: design plan + +This document proposes an end-to-end flow for scanning invoices/bills, extracting text, and turning that into a prefilled transaction. + +## 1) Goal + +Allow users to: + +1. Capture or import an invoice/bill image. +2. Extract readable text from it. +3. Reuse the existing backend API that converts text into transaction data. +4. Open the existing transaction form with prefilled values for user confirmation. +5. Save as a normal transaction via `POST /v2/api/transactions`. + +This keeps final user control and avoids silent bookkeeping mistakes. + +--- + +## 2) Product flow (user journey) + +### Entry points + +- Dashboard FAB menu: add a third action, **Scan invoice/bill**. +- Transaction list screen (optional): toolbar action for scan. + +### Happy path + +1. User taps **Scan invoice/bill**. +2. User chooses **Camera** or **Import from gallery/PDF**. +3. App performs OCR and shows progress. +4. App sends extracted text to the existing backend text-to-transaction endpoint. +5. App opens a **Review draft** step: + - amount + - date + - merchant/payee + - description + - suggested type (expense/income/transfer) + - currency + - confidence and missing fields +6. User confirms or edits fields. +7. App opens `TransactionFormScreen` prefilled. +8. User taps **Create transaction** (existing behavior). + +### Failure path + +- If OCR fails: user can retry capture/import or switch to manual transaction. +- If extraction API fails: keep OCR text visible and allow manual entry with text copy support. +- If confidence is low: flag uncertain fields and require manual confirmation. + +--- + +## 3) Architecture fit (Clean Architecture) + +## UI layer + +Add: + +- `InvoiceScanScreen` (capture/import, progress, retry) +- `InvoiceScanReviewScreen` (draft review and corrections) +- Navigation route(s): + - `transaction/scan` + - `transaction/scan/review` + +Reuse: + +- Existing `TransactionFormScreen` for final creation. +- Existing account pickers/autocomplete for source/destination corrections. + +## Domain layer + +Add use cases: + +- `ExtractTextFromDocumentUseCase` (OCR abstraction) +- `ExtractTransactionFromTextUseCase` (calls existing backend extraction API) +- `BuildTransactionDraftUseCase` (maps OCR + API output to form state) + +Add models: + +- `ScannedDocument` (uri/pages, optional image hash, source) +- `OcrResult` (fullText, line blocks, locale hint) +- `TransactionExtractionDraft` (prefill candidates + confidence per field) + +## Data layer + +Add repository interfaces + implementations: + +- `DocumentTextExtractor`: + - on-device OCR engine (ML Kit Text Recognition) +- `TransactionExtractionRepository`: + - Retrofit call to existing backend text-extraction endpoint + +Keep image handling local by default. Send text only unless backend image upload becomes mandatory. + +--- + +## 4) API integration plan + +The backend capability already exists: "extract transaction information from text". +The Android client should integrate it as a draft-generation step, not an auto-save step. + +### Request payload (proposed) + +Use extracted OCR text and optional context: + +- `text` (required) +- `locale` (optional) +- `defaultCurrency` (optional) +- `timezone` (optional) +- `hints` (optional; e.g., expected type = expense) + +### Response mapping (proposed) + +Expected output fields mapped into existing transaction form state: + +- amount +- currency +- date +- description +- merchant / counterparty name +- transaction type +- optional category/tag suggestions +- per-field confidence + +### Retrofit addition + +Add a new method to `PledgerApiService` for the existing extraction endpoint and wire it through `TransactionRepository` (or a dedicated extraction repository) so ViewModels stay thin. + +--- + +## 5) OCR strategy + +### Phase 1 choice: on-device OCR first + +Use ML Kit text recognition locally: + +- Better privacy (no image upload) +- Fast feedback +- Works with flaky connectivity (only extraction API call needs network) + +### Preprocessing for quality + +Before OCR: + +- auto-crop document bounds (if available) +- perspective correction +- grayscale/contrast normalization +- rotate by EXIF + orientation detection + +These steps significantly improve extraction quality on receipts. + +--- + +## 6) Confidence and validation strategy + +Never create transactions automatically from scan results. + +Rules: + +- Always require user confirmation before save. +- Highlight uncertain fields (confidence below threshold). +- If critical fields are missing (`amount`, `date`, or payee/account side), block one-tap continue and prompt completion. +- Keep raw OCR text expandable so user can verify. + +Suggested thresholds: + +- `>= 0.85`: prefill silently +- `0.60 - 0.84`: prefill with warning badge +- `< 0.60`: leave empty + ask user input + +--- + +## 7) UX details + +- Progress states: + - "Preparing image" + - "Reading text" + - "Extracting transaction details" +- Show editable preview cards for key fields before entering full form. +- Keep "Manual entry instead" action visible at all times. +- Persist last scan draft in `SavedStateHandle` during process death to avoid user frustration. + +--- + +## 8) Security and privacy + +- Default mode: images remain on device; send text only to backend. +- Do not store raw document images in Room. +- If temporary files are created, delete them after review/save/cancel. +- Redact sensitive numbers in logs. +- Respect existing session auth pipeline (JWT via `AuthInterceptor`). + +--- + +## 9) Telemetry and quality metrics + +Track (locally or analytics backend if enabled): + +- scan started/completed +- OCR success rate +- extraction API success rate +- percentage of drafts accepted with no edits +- average number of edited fields before save +- drop-off step (scan, extraction, review, create) + +These metrics identify whether OCR or mapping is the bottleneck. + +--- + +## 10) Test plan + +### Unit tests + +- `BuildTransactionDraftUseCase` mapping and confidence logic. +- Validation rules for required fields. +- Fallback behavior when extraction API errors. + +### Integration tests + +- ViewModel state progression (idle -> scanning -> extracted -> review -> form). +- Retry flow after OCR/API failure. + +### UI tests + +- End-to-end happy path with mocked OCR/API. +- Low-confidence highlighting and manual correction path. + +--- + +## 11) Delivery phases + +### Phase A (MVP) + +- Scan/import single image. +- Local OCR. +- Call existing text extraction endpoint. +- Review screen. +- Prefill and handoff to current `TransactionFormScreen`. + +### Phase B (quality) + +- Better preprocessing/crop. +- Multi-page document support. +- Improved confidence display and field-level explanations. + +### Phase C (advanced) + +- Optional PDF import and page selection. +- Learning from user corrections (future backend support). +- Optional category/tag auto-apply when confidence is high. + +--- + +## 12) Open decisions + +1. Confirm backend endpoint path and DTO schema for text extraction in Android client. +2. Decide whether scan entry belongs only in dashboard FAB or also transaction list. +3. Confirm whether PDF scanning is MVP or Phase B. +4. Confirm privacy policy wording for OCR + text extraction processing. From 22f1c021b0ad458d736bf325081ffe3c9b003d2c Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Mon, 18 May 2026 09:00:10 +0000 Subject: [PATCH 2/8] Implement invoice scan MVP flow and transaction prefill Co-authored-by: Gerben Jongerius --- app/build.gradle.kts | 1 + .../app/data/ocr/InvoiceTextExtractor.kt | 49 ++++ .../app/data/remote/api/PledgerApiService.kt | 5 + .../repository/TransactionRepositoryImpl.kt | 74 ++++++ .../model/TransactionExtractionDraft.kt | 15 ++ .../repository/TransactionRepository.kt | 2 + .../app/ui/dashboard/DashboardAddFabMenu.kt | 22 +- .../app/ui/dashboard/DashboardScreen.kt | 2 + .../pledgerio/app/ui/navigation/NavGraph.kt | 61 ++++- .../com/pledgerio/app/ui/navigation/Screen.kt | 37 ++- .../transactions/TransactionFormViewModel.kt | 104 +++++++++ .../ui/transactions/scan/InvoiceScanScreen.kt | 220 ++++++++++++++++++ .../transactions/scan/InvoiceScanViewModel.kt | 126 ++++++++++ app/src/main/res/values/strings.xml | 15 ++ gradle/libs.versions.toml | 2 + 15 files changed, 727 insertions(+), 8 deletions(-) create mode 100644 app/src/main/java/com/pledgerio/app/data/ocr/InvoiceTextExtractor.kt create mode 100644 app/src/main/java/com/pledgerio/app/domain/model/TransactionExtractionDraft.kt create mode 100644 app/src/main/java/com/pledgerio/app/ui/transactions/scan/InvoiceScanScreen.kt create mode 100644 app/src/main/java/com/pledgerio/app/ui/transactions/scan/InvoiceScanViewModel.kt diff --git a/app/build.gradle.kts b/app/build.gradle.kts index e0d03c6..e752b43 100644 --- a/app/build.gradle.kts +++ b/app/build.gradle.kts @@ -106,6 +106,7 @@ dependencies { implementation(libs.vico.compose.m3) implementation(libs.coil.compose) implementation(libs.androidx.core.splashscreen) + implementation(libs.mlkit.text.recognition) implementation(libs.androidx.compose.ui.text.google.fonts) testImplementation(libs.junit) diff --git a/app/src/main/java/com/pledgerio/app/data/ocr/InvoiceTextExtractor.kt b/app/src/main/java/com/pledgerio/app/data/ocr/InvoiceTextExtractor.kt new file mode 100644 index 0000000..9e03fb7 --- /dev/null +++ b/app/src/main/java/com/pledgerio/app/data/ocr/InvoiceTextExtractor.kt @@ -0,0 +1,49 @@ +package com.pledgerio.app.data.ocr + +import android.content.Context +import android.net.Uri +import com.google.android.gms.tasks.Task +import com.google.mlkit.vision.common.InputImage +import com.google.mlkit.vision.text.TextRecognition +import com.google.mlkit.vision.text.latin.TextRecognizerOptions +import com.pledgerio.app.util.Resource +import dagger.hilt.android.qualifiers.ApplicationContext +import javax.inject.Inject +import javax.inject.Singleton +import kotlinx.coroutines.cancel +import kotlinx.coroutines.suspendCancellableCoroutine +import kotlin.coroutines.resume + +@Singleton +class InvoiceTextExtractor @Inject constructor( + @ApplicationContext private val context: Context, +) { + suspend fun extractText(uri: Uri): Resource { + return try { + val image = InputImage.fromFilePath(context, uri) + val recognizer = TextRecognition.getClient(TextRecognizerOptions.DEFAULT_OPTIONS) + val result = recognizer.process(image).awaitResult() + val text = result.text.trim() + recognizer.close() + if (text.isBlank()) { + Resource.Error("No readable text found. Try a clearer photo.") + } else { + Resource.Success(text) + } + } catch (e: Exception) { + Resource.Error(e.message ?: "Could not read text from the selected image") + } + } +} + +private suspend fun Task.awaitResult(): T = suspendCancellableCoroutine { continuation -> + addOnSuccessListener { result -> + if (continuation.isActive) continuation.resume(result) + } + addOnFailureListener { error -> + if (continuation.isActive) continuation.resumeWith(Result.failure(error)) + } + addOnCanceledListener { + if (continuation.isActive) continuation.cancel() + } +} diff --git a/app/src/main/java/com/pledgerio/app/data/remote/api/PledgerApiService.kt b/app/src/main/java/com/pledgerio/app/data/remote/api/PledgerApiService.kt index fd5548a..903becd 100644 --- a/app/src/main/java/com/pledgerio/app/data/remote/api/PledgerApiService.kt +++ b/app/src/main/java/com/pledgerio/app/data/remote/api/PledgerApiService.kt @@ -109,6 +109,11 @@ interface PledgerApiService { @Query("destination") destination: String? = null, ): Response + @POST("v2/api/ai/extract-transaction") + suspend fun extractTransactionFromText( + @Body request: Map, + ): Response> + // Categories @GET("v2/api/categories") suspend fun getCategories( diff --git a/app/src/main/java/com/pledgerio/app/data/repository/TransactionRepositoryImpl.kt b/app/src/main/java/com/pledgerio/app/data/repository/TransactionRepositoryImpl.kt index 5986708..c285616 100644 --- a/app/src/main/java/com/pledgerio/app/data/repository/TransactionRepositoryImpl.kt +++ b/app/src/main/java/com/pledgerio/app/data/repository/TransactionRepositoryImpl.kt @@ -9,6 +9,7 @@ import com.pledgerio.app.data.remote.dto.TransactionSplitDto import com.pledgerio.app.domain.model.TransactionSplit import com.pledgerio.app.domain.model.Transaction import com.pledgerio.app.domain.model.TransactionClassificationSuggestion +import com.pledgerio.app.domain.model.TransactionExtractionDraft import com.pledgerio.app.domain.model.TransactionFilters import com.pledgerio.app.domain.model.TransactionType import com.pledgerio.app.domain.repository.PagedResult @@ -235,6 +236,46 @@ class TransactionRepositoryImpl @Inject constructor( } } + override suspend fun extractTransactionFromText(text: String): Resource { + if (text.isBlank()) return Resource.Error("No text found in document") + return try { + val response = apiService.extractTransactionFromText( + request = mapOf("text" to text), + ) + if (response.isSuccessful) { + val body = response.body().orEmpty() + val sourceMap = body.readMap("source") + val targetMap = body.readMap("target") + val metadata = body.readMap("metadata") + val rawDate = body.readString("date") ?: body.readString("transactionDate") + Resource.Success( + TransactionExtractionDraft( + description = body.readString("description") ?: body.readString("title"), + amount = body.readDouble("amount") ?: metadata.readDouble("amount"), + currency = body.readString("currency"), + date = rawDate?.let(::parseLocalDateOrNull), + type = body.readString("type")?.let(::parseExtractionType), + sourceName = body.readString("sourceName") + ?: sourceMap.readString("name") + ?: sourceMap.readString("label"), + targetName = body.readString("targetName") + ?: body.readString("counterparty") + ?: targetMap.readString("name") + ?: targetMap.readString("label") + ?: body.readString("merchant"), + confidence = body.readDouble("confidence") + ?: metadata.readDouble("confidence"), + rawText = text, + ), + ) + } else { + Resource.Error("Failed to extract transaction from text: ${response.code()}") + } + } catch (e: Exception) { + Resource.Error(e.message ?: "Network error") + } + } + override fun getRecentTransactions(limit: Int): Flow>> = flow { emit(Resource.Loading) try { @@ -286,4 +327,37 @@ class TransactionRepositoryImpl @Inject constructor( ?: emptyList(), ) } + + private fun parseLocalDateOrNull(value: String): LocalDate? { + return runCatching { LocalDate.parse(value) }.getOrNull() + } + + private fun parseExtractionType(value: String): TransactionType? { + return when (value.trim().uppercase()) { + "CREDIT", "EXPENSE" -> TransactionType.CREDIT + "DEBIT", "INCOME" -> TransactionType.DEBIT + "TRANSFER" -> TransactionType.TRANSFER + else -> null + } + } + + private fun Map.readString(key: String): String? { + val value = this[key] ?: return null + return (value as? String)?.trim()?.takeIf { it.isNotBlank() } + } + + private fun Map.readDouble(key: String): Double? { + val value = this[key] ?: return null + return when (value) { + is Number -> value.toDouble() + is String -> value.toDoubleOrNull() + else -> null + } + } + + private fun Map.readMap(key: String): Map { + val value = this[key] ?: return emptyMap() + @Suppress("UNCHECKED_CAST") + return value as? Map ?: emptyMap() + } } diff --git a/app/src/main/java/com/pledgerio/app/domain/model/TransactionExtractionDraft.kt b/app/src/main/java/com/pledgerio/app/domain/model/TransactionExtractionDraft.kt new file mode 100644 index 0000000..219a24b --- /dev/null +++ b/app/src/main/java/com/pledgerio/app/domain/model/TransactionExtractionDraft.kt @@ -0,0 +1,15 @@ +package com.pledgerio.app.domain.model + +import java.time.LocalDate + +data class TransactionExtractionDraft( + val description: String? = null, + val amount: Double? = null, + val currency: String? = null, + val date: LocalDate? = null, + val type: TransactionType? = null, + val sourceName: String? = null, + val targetName: String? = null, + val confidence: Double? = null, + val rawText: String? = null, +) diff --git a/app/src/main/java/com/pledgerio/app/domain/repository/TransactionRepository.kt b/app/src/main/java/com/pledgerio/app/domain/repository/TransactionRepository.kt index 17b5220..2330b58 100644 --- a/app/src/main/java/com/pledgerio/app/domain/repository/TransactionRepository.kt +++ b/app/src/main/java/com/pledgerio/app/domain/repository/TransactionRepository.kt @@ -2,6 +2,7 @@ package com.pledgerio.app.domain.repository import com.pledgerio.app.domain.model.Transaction import com.pledgerio.app.domain.model.TransactionClassificationSuggestion +import com.pledgerio.app.domain.model.TransactionExtractionDraft import com.pledgerio.app.domain.model.TransactionFilters import com.pledgerio.app.domain.model.TransactionSplit import com.pledgerio.app.domain.model.TransactionType @@ -42,5 +43,6 @@ interface TransactionRepository { source: String? = null, destination: String? = null, ): Resource + suspend fun extractTransactionFromText(text: String): Resource fun getRecentTransactions(limit: Int = 5): Flow>> } diff --git a/app/src/main/java/com/pledgerio/app/ui/dashboard/DashboardAddFabMenu.kt b/app/src/main/java/com/pledgerio/app/ui/dashboard/DashboardAddFabMenu.kt index 516c135..2b6affa 100644 --- a/app/src/main/java/com/pledgerio/app/ui/dashboard/DashboardAddFabMenu.kt +++ b/app/src/main/java/com/pledgerio/app/ui/dashboard/DashboardAddFabMenu.kt @@ -26,6 +26,7 @@ import androidx.compose.material.icons.Icons import androidx.compose.material.icons.filled.AccountBalance import androidx.compose.material.icons.filled.Add import androidx.compose.material.icons.filled.Receipt +import androidx.compose.material.icons.filled.DocumentScanner import androidx.compose.material3.Card import androidx.compose.material3.CardDefaults import androidx.compose.material3.FloatingActionButton @@ -42,7 +43,9 @@ import androidx.compose.ui.draw.rotate import androidx.compose.ui.graphics.Color import androidx.compose.ui.graphics.vector.ImageVector import androidx.compose.ui.text.font.FontWeight +import androidx.compose.ui.res.stringResource import androidx.compose.ui.unit.dp +import com.pledgerio.app.R import com.pledgerio.app.ui.theme.EmeraldGreen @Composable @@ -50,6 +53,7 @@ fun DashboardAddFabMenu( expanded: Boolean, onExpandedChange: (Boolean) -> Unit, onAddTransaction: () -> Unit, + onScanInvoice: () -> Unit, onAddAccount: () -> Unit, modifier: Modifier = Modifier, ) { @@ -104,18 +108,28 @@ fun DashboardAddFabMenu( AddMenuActionRow( icon = Icons.Default.Receipt, iconTint = EmeraldGreen, - title = "New transaction", - subtitle = "Record income, expense or transfer", + title = stringResource(R.string.fab_new_transaction), + subtitle = stringResource(R.string.fab_new_transaction_subtitle), onClick = { onExpandedChange(false) onAddTransaction() }, ) + AddMenuActionRow( + icon = Icons.Default.DocumentScanner, + iconTint = MaterialTheme.colorScheme.primary, + title = stringResource(R.string.fab_scan_invoice), + subtitle = stringResource(R.string.fab_scan_invoice_subtitle), + onClick = { + onExpandedChange(false) + onScanInvoice() + }, + ) AddMenuActionRow( icon = Icons.Default.AccountBalance, iconTint = MaterialTheme.colorScheme.tertiary, - title = "New account", - subtitle = "Add a bank account or wallet", + title = stringResource(R.string.fab_new_account), + subtitle = stringResource(R.string.fab_new_account_subtitle), onClick = { onExpandedChange(false) onAddAccount() diff --git a/app/src/main/java/com/pledgerio/app/ui/dashboard/DashboardScreen.kt b/app/src/main/java/com/pledgerio/app/ui/dashboard/DashboardScreen.kt index bbcd415..be214a9 100644 --- a/app/src/main/java/com/pledgerio/app/ui/dashboard/DashboardScreen.kt +++ b/app/src/main/java/com/pledgerio/app/ui/dashboard/DashboardScreen.kt @@ -76,6 +76,7 @@ fun DashboardScreen( onNavigateToTransactions: () -> Unit, onNavigateToTransaction: (Long) -> Unit, onNavigateToAddTransaction: () -> Unit, + onNavigateToScanInvoice: () -> Unit, onNavigateToAddAccount: () -> Unit, onNavigateToSettings: () -> Unit, onNavigateToSearch: () -> Unit = {}, @@ -230,6 +231,7 @@ fun DashboardScreen( expanded = showAddMenu, onExpandedChange = { showAddMenu = it }, onAddTransaction = onNavigateToAddTransaction, + onScanInvoice = onNavigateToScanInvoice, onAddAccount = onNavigateToAddAccount, ) } diff --git a/app/src/main/java/com/pledgerio/app/ui/navigation/NavGraph.kt b/app/src/main/java/com/pledgerio/app/ui/navigation/NavGraph.kt index a759604..ae84d73 100644 --- a/app/src/main/java/com/pledgerio/app/ui/navigation/NavGraph.kt +++ b/app/src/main/java/com/pledgerio/app/ui/navigation/NavGraph.kt @@ -27,6 +27,7 @@ import com.pledgerio.app.ui.settings.SettingsScreen import com.pledgerio.app.ui.transactions.TransactionDetailScreen import com.pledgerio.app.ui.transactions.TransactionFormScreen import com.pledgerio.app.ui.transactions.TransactionsScreen +import com.pledgerio.app.ui.transactions.scan.InvoiceScanScreen @Composable fun NavGraph( @@ -88,7 +89,10 @@ fun NavGraph( navController.navigate(Screen.TransactionDetail.createRoute(id)) }, onNavigateToAddTransaction = { - navController.navigate(Screen.AddTransaction.route) + navController.navigate(Screen.AddTransaction.createRoute()) + }, + onNavigateToScanInvoice = { + navController.navigate(Screen.InvoiceScan.route) }, onNavigateToAddAccount = { navController.navigate(Screen.AddAccount.createRoute()) @@ -132,7 +136,7 @@ fun NavGraph( navController.navigate(Screen.TransactionDetail.createRoute(id)) }, onNavigateToAdd = { - navController.navigate(Screen.AddTransaction.route) + navController.navigate(Screen.AddTransaction.createRoute()) }, onNavigateToSettings = { navController.navigate(Screen.Settings.route) @@ -140,7 +144,39 @@ fun NavGraph( ) } - composable(Screen.AddTransaction.route) { + composable( + route = Screen.AddTransaction.route, + arguments = listOf( + navArgument("prefillDescription") { + type = NavType.StringType + defaultValue = "" + }, + navArgument("prefillAmount") { + type = NavType.StringType + defaultValue = "" + }, + navArgument("prefillCurrency") { + type = NavType.StringType + defaultValue = "" + }, + navArgument("prefillDate") { + type = NavType.StringType + defaultValue = "" + }, + navArgument("prefillType") { + type = NavType.StringType + defaultValue = "" + }, + navArgument("prefillSource") { + type = NavType.StringType + defaultValue = "" + }, + navArgument("prefillTarget") { + type = NavType.StringType + defaultValue = "" + }, + ), + ) { TransactionFormScreen( onNavigateBack = { navController.popBackStack() }, onNavigateToAddAccount = { typeCode -> @@ -149,6 +185,25 @@ fun NavGraph( ) } + composable(Screen.InvoiceScan.route) { + InvoiceScanScreen( + onNavigateBack = { navController.popBackStack() }, + onUseDraft = { draft -> + navController.navigate( + Screen.AddTransaction.createRoute( + prefillDescription = draft.description, + prefillAmount = draft.amount?.toString(), + prefillCurrency = draft.currency, + prefillDate = draft.date?.toString(), + prefillType = draft.type?.name, + prefillSource = draft.sourceName, + prefillTarget = draft.targetName, + ), + ) + }, + ) + } + composable( route = Screen.EditTransaction.route, arguments = listOf(navArgument("transactionId") { type = NavType.LongType }), diff --git a/app/src/main/java/com/pledgerio/app/ui/navigation/Screen.kt b/app/src/main/java/com/pledgerio/app/ui/navigation/Screen.kt index b0e2ffe..fc3d655 100644 --- a/app/src/main/java/com/pledgerio/app/ui/navigation/Screen.kt +++ b/app/src/main/java/com/pledgerio/app/ui/navigation/Screen.kt @@ -1,5 +1,7 @@ package com.pledgerio.app.ui.navigation +import android.net.Uri + sealed class Screen(val route: String) { data object ServerSetup : Screen("server_setup?changeServer={changeServer}") { fun createRoute(changeServer: Boolean = false) = @@ -17,7 +19,40 @@ sealed class Screen(val route: String) { data object TransactionDetail : Screen("transaction/{transactionId}") { fun createRoute(transactionId: Long) = "transaction/$transactionId" } - data object AddTransaction : Screen("transaction/add") + data object AddTransaction : Screen( + "transaction/add?" + + "prefillDescription={prefillDescription}&" + + "prefillAmount={prefillAmount}&" + + "prefillCurrency={prefillCurrency}&" + + "prefillDate={prefillDate}&" + + "prefillType={prefillType}&" + + "prefillSource={prefillSource}&" + + "prefillTarget={prefillTarget}" + ) { + fun createRoute( + prefillDescription: String? = null, + prefillAmount: String? = null, + prefillCurrency: String? = null, + prefillDate: String? = null, + prefillType: String? = null, + prefillSource: String? = null, + prefillTarget: String? = null, + ): String { + val params = listOf( + "prefillDescription" to prefillDescription, + "prefillAmount" to prefillAmount, + "prefillCurrency" to prefillCurrency, + "prefillDate" to prefillDate, + "prefillType" to prefillType, + "prefillSource" to prefillSource, + "prefillTarget" to prefillTarget, + ).joinToString("&") { (key, value) -> + "$key=${Uri.encode(value.orEmpty())}" + } + return "transaction/add?$params" + } + } + data object InvoiceScan : Screen("transaction/scan") data object EditTransaction : Screen("transaction/{transactionId}/edit") { fun createRoute(transactionId: Long) = "transaction/$transactionId/edit" } diff --git a/app/src/main/java/com/pledgerio/app/ui/transactions/TransactionFormViewModel.kt b/app/src/main/java/com/pledgerio/app/ui/transactions/TransactionFormViewModel.kt index aaa8f6f..175b4a4 100644 --- a/app/src/main/java/com/pledgerio/app/ui/transactions/TransactionFormViewModel.kt +++ b/app/src/main/java/com/pledgerio/app/ui/transactions/TransactionFormViewModel.kt @@ -173,6 +173,16 @@ data class TransactionFormUiState( } } +private data class PrefillDraft( + val description: String?, + val amount: String?, + val currency: String?, + val date: LocalDate?, + val type: TransactionType?, + val sourceName: String?, + val targetName: String?, +) + @OptIn(FlowPreview::class, kotlinx.coroutines.ExperimentalCoroutinesApi::class) @HiltViewModel class TransactionFormViewModel @Inject constructor( @@ -189,6 +199,11 @@ class TransactionFormViewModel @Inject constructor( ) : ViewModel() { private val editTransactionId: Long? = savedStateHandle.get("transactionId") + private val prefillDraft: PrefillDraft? = if (editTransactionId == null) { + savedStateHandle.toPrefillDraft() + } else { + null + } private val _uiState = MutableStateFlow( TransactionFormUiState( @@ -1332,6 +1347,7 @@ class TransactionFormViewModel @Inject constructor( _uiState.update { it.copy(ownedAccounts = result.data) } if (editTransactionId == null) { applyLastTransactionType() + applyPrefillIfPresent() } else { _uiState.update { it.copy(isLoading = false) } } @@ -1356,6 +1372,47 @@ class TransactionFormViewModel @Inject constructor( } } + private fun applyPrefillIfPresent() { + val prefill = prefillDraft ?: return + _uiState.update { state -> + val type = prefill.type ?: state.type + state.copy( + type = type, + description = prefill.description ?: state.description, + amount = prefill.amount ?: state.amount, + currency = prefill.currency + ?.takeIf { state.currencies.contains(it) } + ?: state.currency, + date = prefill.date ?: state.date, + sourceAccountId = null, + sourceSelected = null, + sourceQuery = prefill.sourceName.orEmpty(), + sourceSuggestions = emptyList(), + targetAccountId = null, + targetSelected = null, + targetQuery = prefill.targetName.orEmpty(), + targetSuggestions = emptyList(), + error = null, + ) + } + + viewModelScope.launch { + val current = _uiState.value + if ( + current.sourceInputKind != AccountInputKind.OWNED_DROPDOWN && + current.sourceQuery.isNotBlank() + ) { + searchSourceAccounts(current.sourceQuery) + } + if ( + current.targetInputKind != AccountInputKind.OWNED_DROPDOWN && + current.targetQuery.isNotBlank() + ) { + searchTargetAccounts(current.targetQuery) + } + } + } + private fun observeExperienceMode() { viewModelScope.launch { userPreferences.financeExperienceMode.collect { mode -> @@ -1423,3 +1480,50 @@ internal fun List.toDomainSplits(): List("prefillDescription")?.trim().orEmpty().ifBlank { null } + val amount = get("prefillAmount")?.trim().orEmpty().ifBlank { null } + val currency = get("prefillCurrency")?.trim().orEmpty().ifBlank { null } + val date = get("prefillDate") + ?.trim() + ?.takeIf { it.isNotBlank() } + ?.let { value -> runCatching { LocalDate.parse(value) }.getOrNull() } + val type = get("prefillType") + ?.trim() + ?.takeIf { it.isNotBlank() } + ?.let(::toTransactionTypeOrNull) + val source = get("prefillSource")?.trim().orEmpty().ifBlank { null } + val target = get("prefillTarget")?.trim().orEmpty().ifBlank { null } + + return if ( + description == null && + amount == null && + currency == null && + date == null && + type == null && + source == null && + target == null + ) { + null + } else { + PrefillDraft( + description = description, + amount = amount, + currency = currency, + date = date, + type = type, + sourceName = source, + targetName = target, + ) + } +} + +private fun toTransactionTypeOrNull(raw: String): TransactionType? { + return when (raw.uppercase()) { + "CREDIT", "EXPENSE" -> TransactionType.CREDIT + "DEBIT", "INCOME" -> TransactionType.DEBIT + "TRANSFER" -> TransactionType.TRANSFER + else -> null + } +} diff --git a/app/src/main/java/com/pledgerio/app/ui/transactions/scan/InvoiceScanScreen.kt b/app/src/main/java/com/pledgerio/app/ui/transactions/scan/InvoiceScanScreen.kt new file mode 100644 index 0000000..460b90a --- /dev/null +++ b/app/src/main/java/com/pledgerio/app/ui/transactions/scan/InvoiceScanScreen.kt @@ -0,0 +1,220 @@ +package com.pledgerio.app.ui.transactions.scan + +import androidx.activity.compose.rememberLauncherForActivityResult +import androidx.activity.result.contract.ActivityResultContracts +import androidx.compose.foundation.layout.Arrangement +import androidx.compose.foundation.layout.Column +import androidx.compose.foundation.layout.Row +import androidx.compose.foundation.layout.Spacer +import androidx.compose.foundation.layout.fillMaxSize +import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.foundation.layout.height +import androidx.compose.foundation.layout.padding +import androidx.compose.foundation.rememberScrollState +import androidx.compose.foundation.verticalScroll +import androidx.compose.material.icons.Icons +import androidx.compose.material.icons.automirrored.filled.ArrowBack +import androidx.compose.material.icons.filled.Image +import androidx.compose.material3.Button +import androidx.compose.material3.Icon +import androidx.compose.material3.IconButton +import androidx.compose.material3.LinearProgressIndicator +import androidx.compose.material3.MaterialTheme +import androidx.compose.material3.OutlinedTextField +import androidx.compose.material3.Scaffold +import androidx.compose.material3.Text +import androidx.compose.runtime.Composable +import androidx.compose.runtime.collectAsState +import androidx.compose.runtime.getValue +import androidx.compose.ui.Modifier +import androidx.compose.ui.layout.ContentScale +import androidx.compose.ui.res.stringResource +import androidx.compose.ui.unit.dp +import androidx.hilt.navigation.compose.hiltViewModel +import coil.compose.AsyncImage +import com.pledgerio.app.R +import com.pledgerio.app.domain.model.TransactionExtractionDraft +import com.pledgerio.app.ui.components.PledgerCard +import com.pledgerio.app.ui.components.PledgerTopBar +import com.pledgerio.app.util.formatCurrency + +@Composable +fun InvoiceScanScreen( + onNavigateBack: () -> Unit, + onUseDraft: (TransactionExtractionDraft) -> Unit, + viewModel: InvoiceScanViewModel = hiltViewModel(), +) { + val uiState by viewModel.uiState.collectAsState() + val imagePickerLauncher = rememberLauncherForActivityResult( + contract = ActivityResultContracts.GetContent(), + ) { uri -> + if (uri != null) viewModel.onImageSelected(uri) + } + + Scaffold( + topBar = { + PledgerTopBar( + title = stringResource(R.string.invoice_scan_title), + subtitle = stringResource(R.string.invoice_scan_subtitle), + navigationIcon = { + IconButton(onClick = onNavigateBack) { + Icon( + imageVector = Icons.AutoMirrored.Filled.ArrowBack, + contentDescription = stringResource(R.string.nav_back), + ) + } + }, + ) + }, + ) { paddingValues -> + Column( + modifier = Modifier + .fillMaxSize() + .padding(paddingValues) + .padding(horizontal = 16.dp) + .verticalScroll(rememberScrollState()), + verticalArrangement = Arrangement.spacedBy(12.dp), + ) { + Spacer(modifier = Modifier.height(4.dp)) + + Button( + onClick = { imagePickerLauncher.launch("image/*") }, + modifier = Modifier.fillMaxWidth(), + enabled = !uiState.isWorking, + ) { + Icon( + imageVector = Icons.Default.Image, + contentDescription = null, + ) + Text( + text = stringResource(R.string.invoice_scan_pick_image), + modifier = Modifier.padding(start = 8.dp), + ) + } + + if (uiState.isWorking) { + LinearProgressIndicator(modifier = Modifier.fillMaxWidth()) + Text( + text = when (uiState.stage) { + InvoiceScanStage.READING_TEXT -> stringResource(R.string.invoice_scan_reading_text) + InvoiceScanStage.EXTRACTING_TRANSACTION -> stringResource(R.string.invoice_scan_extracting_transaction) + InvoiceScanStage.IDLE -> "" + }, + style = MaterialTheme.typography.bodySmall, + color = MaterialTheme.colorScheme.onSurfaceVariant, + ) + } + + uiState.selectedImageUri?.let { uri -> + PledgerCard { + AsyncImage( + model = uri, + contentDescription = stringResource(R.string.invoice_scan_image_preview), + contentScale = ContentScale.Crop, + modifier = Modifier + .fillMaxWidth() + .height(180.dp), + ) + } + } + + OutlinedTextField( + value = uiState.extractedText, + onValueChange = viewModel::onTextChanged, + label = { Text(stringResource(R.string.invoice_scan_text_label)) }, + placeholder = { Text(stringResource(R.string.invoice_scan_text_placeholder)) }, + minLines = 4, + maxLines = 10, + modifier = Modifier.fillMaxWidth(), + ) + + Row( + horizontalArrangement = Arrangement.spacedBy(12.dp), + modifier = Modifier.fillMaxWidth(), + ) { + Button( + onClick = viewModel::extractFromCurrentText, + enabled = !uiState.isWorking, + modifier = Modifier.weight(1f), + ) { + Text(stringResource(R.string.invoice_scan_extract_button)) + } + } + + uiState.error?.takeIf { it.isNotBlank() }?.let { message -> + Text( + text = message, + style = MaterialTheme.typography.bodyMedium, + color = MaterialTheme.colorScheme.error, + ) + } + + uiState.draft?.let { draft -> + DraftPreviewCard( + draft = draft, + onUseDraft = { onUseDraft(draft) }, + ) + } + + Spacer(modifier = Modifier.height(16.dp)) + } + } +} + +@Composable +private fun DraftPreviewCard( + draft: TransactionExtractionDraft, + onUseDraft: () -> Unit, +) { + PledgerCard { + Text( + text = stringResource(R.string.invoice_scan_preview_title), + style = MaterialTheme.typography.titleMedium, + ) + Spacer(modifier = Modifier.height(8.dp)) + DraftLine( + label = stringResource(R.string.transaction_description_label), + value = draft.description, + ) + DraftLine( + label = stringResource(R.string.transaction_amount_label), + value = draft.amount?.formatCurrency(draft.currency ?: "EUR"), + ) + DraftLine( + label = stringResource(R.string.transaction_date_label), + value = draft.date?.toString(), + ) + DraftLine( + label = stringResource(R.string.invoice_scan_preview_counterparty), + value = draft.targetName ?: draft.sourceName, + ) + DraftLine( + label = stringResource(R.string.invoice_scan_preview_confidence), + value = draft.confidence?.let { String.format("%.0f%%", it * 100) }, + ) + Spacer(modifier = Modifier.height(12.dp)) + Button( + onClick = onUseDraft, + modifier = Modifier.fillMaxWidth(), + ) { + Text(stringResource(R.string.invoice_scan_use_draft_button)) + } + } +} + +@Composable +private fun DraftLine( + label: String, + value: String?, +) { + Text( + text = label, + style = MaterialTheme.typography.labelMedium, + color = MaterialTheme.colorScheme.onSurfaceVariant, + ) + Text( + text = value?.takeIf { it.isNotBlank() } ?: stringResource(R.string.settings_not_configured), + style = MaterialTheme.typography.bodyMedium, + ) + Spacer(modifier = Modifier.height(6.dp)) +} diff --git a/app/src/main/java/com/pledgerio/app/ui/transactions/scan/InvoiceScanViewModel.kt b/app/src/main/java/com/pledgerio/app/ui/transactions/scan/InvoiceScanViewModel.kt new file mode 100644 index 0000000..c58d55c --- /dev/null +++ b/app/src/main/java/com/pledgerio/app/ui/transactions/scan/InvoiceScanViewModel.kt @@ -0,0 +1,126 @@ +package com.pledgerio.app.ui.transactions.scan + +import android.net.Uri +import androidx.lifecycle.ViewModel +import androidx.lifecycle.viewModelScope +import com.pledgerio.app.data.ocr.InvoiceTextExtractor +import com.pledgerio.app.domain.model.TransactionExtractionDraft +import com.pledgerio.app.domain.repository.TransactionRepository +import com.pledgerio.app.util.Resource +import dagger.hilt.android.lifecycle.HiltViewModel +import javax.inject.Inject +import kotlinx.coroutines.flow.MutableStateFlow +import kotlinx.coroutines.flow.StateFlow +import kotlinx.coroutines.flow.asStateFlow +import kotlinx.coroutines.flow.update +import kotlinx.coroutines.launch + +enum class InvoiceScanStage { + IDLE, + READING_TEXT, + EXTRACTING_TRANSACTION, +} + +data class InvoiceScanUiState( + val stage: InvoiceScanStage = InvoiceScanStage.IDLE, + val selectedImageUri: Uri? = null, + val extractedText: String = "", + val draft: TransactionExtractionDraft? = null, + val error: String? = null, +) { + val isWorking: Boolean + get() = stage != InvoiceScanStage.IDLE +} + +@HiltViewModel +class InvoiceScanViewModel @Inject constructor( + private val invoiceTextExtractor: InvoiceTextExtractor, + private val transactionRepository: TransactionRepository, +) : ViewModel() { + + private val _uiState = MutableStateFlow(InvoiceScanUiState()) + val uiState: StateFlow = _uiState.asStateFlow() + + fun onTextChanged(value: String) { + _uiState.update { + it.copy( + extractedText = value, + error = null, + draft = null, + ) + } + } + + fun onImageSelected(uri: Uri) { + viewModelScope.launch { + _uiState.update { + it.copy( + selectedImageUri = uri, + stage = InvoiceScanStage.READING_TEXT, + error = null, + draft = null, + ) + } + when (val ocrResult = invoiceTextExtractor.extractText(uri)) { + is Resource.Success -> { + _uiState.update { + it.copy( + extractedText = ocrResult.data, + stage = InvoiceScanStage.EXTRACTING_TRANSACTION, + ) + } + extractFromCurrentText() + } + + is Resource.Error -> { + _uiState.update { + it.copy( + stage = InvoiceScanStage.IDLE, + error = ocrResult.message, + ) + } + } + + is Resource.Loading -> Unit + } + } + } + + fun extractFromCurrentText() { + val text = _uiState.value.extractedText.trim() + if (text.isBlank()) { + _uiState.update { it.copy(error = "No text to extract. Import an invoice photo first.") } + return + } + viewModelScope.launch { + _uiState.update { + it.copy( + stage = InvoiceScanStage.EXTRACTING_TRANSACTION, + error = null, + draft = null, + ) + } + when (val result = transactionRepository.extractTransactionFromText(text)) { + is Resource.Success -> { + _uiState.update { + it.copy( + stage = InvoiceScanStage.IDLE, + draft = result.data.copy(rawText = text), + ) + } + } + + is Resource.Error -> { + _uiState.update { + it.copy( + stage = InvoiceScanStage.IDLE, + error = result.message, + ) + } + } + + is Resource.Loading -> Unit + } + } + } +} diff --git a/app/src/main/res/values/strings.xml b/app/src/main/res/values/strings.xml index 4f9add0..52ed1d6 100644 --- a/app/src/main/res/values/strings.xml +++ b/app/src/main/res/values/strings.xml @@ -180,6 +180,8 @@ Expenses New transaction Record income, expense or transfer + Scan invoice/bill + Extract text and prefill a transaction New account Add a bank account or wallet @@ -405,6 +407,19 @@ No accounts available No matches New template + Scan invoice or bill + Import an image, extract text, and review the draft + Pick invoice image + Reading text from image… + Extracting transaction details… + Selected invoice image + Extracted text + The invoice text will appear here after OCR. + Extract transaction draft + Draft preview + Counterparty + Confidence + Use draft in transaction form No results Clear Collapse diff --git a/gradle/libs.versions.toml b/gradle/libs.versions.toml index a1d783f..0551b7b 100644 --- a/gradle/libs.versions.toml +++ b/gradle/libs.versions.toml @@ -34,6 +34,7 @@ paging = "3.3.6" vico = "2.0.1" coil = "2.7.0" splashscreen = "1.0.1" +mlkitTextRecognition = "16.0.1" junit = "4.13.2" coroutinesTest = "1.10.2" @@ -94,6 +95,7 @@ androidx-paging-compose = { group = "androidx.paging", name = "paging-compose", vico-compose-m3 = { group = "com.patrykandpatrick.vico", name = "compose-m3", version.ref = "vico" } coil-compose = { group = "io.coil-kt", name = "coil-compose", version.ref = "coil" } androidx-core-splashscreen = { group = "androidx.core", name = "core-splashscreen", version.ref = "splashscreen" } +mlkit-text-recognition = { group = "com.google.mlkit", name = "text-recognition", version.ref = "mlkitTextRecognition" } junit = { group = "junit", name = "junit", version.ref = "junit" } kotlinx-coroutines-test = { group = "org.jetbrains.kotlinx", name = "kotlinx-coroutines-test", version.ref = "coroutinesTest" } From 37f55ae3f313031960cd40714760a38651c91319 Mon Sep 17 00:00:00 2001 From: Gerben Jongerius Date: Mon, 18 May 2026 19:31:21 +0200 Subject: [PATCH 3/8] Fix the extract end-point and structure --- .../pledgerio/app/data/remote/api/PledgerApiService.kt | 4 ++-- .../pledgerio/app/data/remote/dto/TransactionExtract.kt | 9 +++++++++ .../app/data/repository/TransactionRepositoryImpl.kt | 3 ++- .../pledgerio/app/ui/accounts/AccountTypePickerTest.kt | 4 +++- 4 files changed, 16 insertions(+), 4 deletions(-) create mode 100644 app/src/main/java/com/pledgerio/app/data/remote/dto/TransactionExtract.kt diff --git a/app/src/main/java/com/pledgerio/app/data/remote/api/PledgerApiService.kt b/app/src/main/java/com/pledgerio/app/data/remote/api/PledgerApiService.kt index 903becd..ab67ac6 100644 --- a/app/src/main/java/com/pledgerio/app/data/remote/api/PledgerApiService.kt +++ b/app/src/main/java/com/pledgerio/app/data/remote/api/PledgerApiService.kt @@ -109,9 +109,9 @@ interface PledgerApiService { @Query("destination") destination: String? = null, ): Response - @POST("v2/api/ai/extract-transaction") + @POST("v2/api/ai/extract") suspend fun extractTransactionFromText( - @Body request: Map, + @Body request: TransactionExtract, ): Response> // Categories diff --git a/app/src/main/java/com/pledgerio/app/data/remote/dto/TransactionExtract.kt b/app/src/main/java/com/pledgerio/app/data/remote/dto/TransactionExtract.kt new file mode 100644 index 0000000..7f53976 --- /dev/null +++ b/app/src/main/java/com/pledgerio/app/data/remote/dto/TransactionExtract.kt @@ -0,0 +1,9 @@ +package com.pledgerio.app.data.remote.dto + +import com.squareup.moshi.Json +import com.squareup.moshi.JsonClass + +@JsonClass(generateAdapter = true) +data class TransactionExtract( + @Json(name = "text") val text: String +) diff --git a/app/src/main/java/com/pledgerio/app/data/repository/TransactionRepositoryImpl.kt b/app/src/main/java/com/pledgerio/app/data/repository/TransactionRepositoryImpl.kt index c285616..f057f54 100644 --- a/app/src/main/java/com/pledgerio/app/data/repository/TransactionRepositoryImpl.kt +++ b/app/src/main/java/com/pledgerio/app/data/repository/TransactionRepositoryImpl.kt @@ -5,6 +5,7 @@ import com.pledgerio.app.data.local.entity.TransactionEntity import com.pledgerio.app.data.remote.api.PledgerApiService import com.pledgerio.app.data.remote.dto.CreateTransactionRequest import com.pledgerio.app.data.remote.dto.TransactionDto +import com.pledgerio.app.data.remote.dto.TransactionExtract import com.pledgerio.app.data.remote.dto.TransactionSplitDto import com.pledgerio.app.domain.model.TransactionSplit import com.pledgerio.app.domain.model.Transaction @@ -240,7 +241,7 @@ class TransactionRepositoryImpl @Inject constructor( if (text.isBlank()) return Resource.Error("No text found in document") return try { val response = apiService.extractTransactionFromText( - request = mapOf("text" to text), + request = TransactionExtract(text), ) if (response.isSuccessful) { val body = response.body().orEmpty() diff --git a/app/src/test/java/com/pledgerio/app/ui/accounts/AccountTypePickerTest.kt b/app/src/test/java/com/pledgerio/app/ui/accounts/AccountTypePickerTest.kt index c1f7125..f468b7b 100644 --- a/app/src/test/java/com/pledgerio/app/ui/accounts/AccountTypePickerTest.kt +++ b/app/src/test/java/com/pledgerio/app/ui/accounts/AccountTypePickerTest.kt @@ -47,7 +47,9 @@ class AccountTypePickerTest { ) assertEquals(2, entries.size) - val card = entries.single { (it.label as AccountPickerLabel.Custom).value == "Credit card" } + val card = entries.single { it.soloTypeCode == "credit_card" } + assertTrue(card.label is AccountPickerLabel.Custom) + assertEquals("Credit card", (card.label as AccountPickerLabel.Custom).value) assertNull(card.jointTypeCode) assertNull(card.family) assertEquals("credit_card", card.soloTypeCode) From 2374939acde8aa6a5dea6e1b497107d43ef1d924 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Mon, 18 May 2026 17:34:58 +0000 Subject: [PATCH 4/8] Polish invoice scan UX with guided step layout Co-authored-by: Gerben Jongerius --- .../pledgerio/app/ui/navigation/NavGraph.kt | 3 + .../ui/transactions/scan/InvoiceScanScreen.kt | 389 +++++++++++++++--- app/src/main/res/values/strings.xml | 15 + 3 files changed, 343 insertions(+), 64 deletions(-) diff --git a/app/src/main/java/com/pledgerio/app/ui/navigation/NavGraph.kt b/app/src/main/java/com/pledgerio/app/ui/navigation/NavGraph.kt index ae84d73..5fac9f4 100644 --- a/app/src/main/java/com/pledgerio/app/ui/navigation/NavGraph.kt +++ b/app/src/main/java/com/pledgerio/app/ui/navigation/NavGraph.kt @@ -201,6 +201,9 @@ fun NavGraph( ), ) }, + onManualEntry = { + navController.navigate(Screen.AddTransaction.createRoute()) + }, ) } diff --git a/app/src/main/java/com/pledgerio/app/ui/transactions/scan/InvoiceScanScreen.kt b/app/src/main/java/com/pledgerio/app/ui/transactions/scan/InvoiceScanScreen.kt index 460b90a..0a8c914 100644 --- a/app/src/main/java/com/pledgerio/app/ui/transactions/scan/InvoiceScanScreen.kt +++ b/app/src/main/java/com/pledgerio/app/ui/transactions/scan/InvoiceScanScreen.kt @@ -2,24 +2,39 @@ package com.pledgerio.app.ui.transactions.scan import androidx.activity.compose.rememberLauncherForActivityResult import androidx.activity.result.contract.ActivityResultContracts +import androidx.compose.foundation.background +import androidx.compose.foundation.border import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.Spacer +import androidx.compose.foundation.layout.WindowInsets +import androidx.compose.foundation.layout.asPaddingValues +import androidx.compose.foundation.layout.defaultMinSize import androidx.compose.foundation.layout.fillMaxSize import androidx.compose.foundation.layout.fillMaxWidth import androidx.compose.foundation.layout.height +import androidx.compose.foundation.layout.navigationBars import androidx.compose.foundation.layout.padding +import androidx.compose.foundation.layout.size import androidx.compose.foundation.rememberScrollState +import androidx.compose.foundation.shape.CircleShape +import androidx.compose.foundation.shape.RoundedCornerShape import androidx.compose.foundation.verticalScroll import androidx.compose.material.icons.Icons import androidx.compose.material.icons.automirrored.filled.ArrowBack +import androidx.compose.material.icons.filled.AutoAwesome +import androidx.compose.material.icons.filled.CalendarMonth +import androidx.compose.material.icons.filled.Description import androidx.compose.material.icons.filled.Image +import androidx.compose.material.icons.filled.Person +import androidx.compose.material.icons.filled.Sell import androidx.compose.material3.Button import androidx.compose.material3.Icon import androidx.compose.material3.IconButton import androidx.compose.material3.LinearProgressIndicator import androidx.compose.material3.MaterialTheme +import androidx.compose.material3.OutlinedButton import androidx.compose.material3.OutlinedTextField import androidx.compose.material3.Scaffold import androidx.compose.material3.Text @@ -27,8 +42,12 @@ import androidx.compose.runtime.Composable import androidx.compose.runtime.collectAsState import androidx.compose.runtime.getValue import androidx.compose.ui.Modifier +import androidx.compose.ui.draw.clip +import androidx.compose.ui.graphics.Color +import androidx.compose.ui.graphics.vector.ImageVector import androidx.compose.ui.layout.ContentScale import androidx.compose.ui.res.stringResource +import androidx.compose.ui.text.font.FontWeight import androidx.compose.ui.unit.dp import androidx.hilt.navigation.compose.hiltViewModel import coil.compose.AsyncImage @@ -37,14 +56,22 @@ import com.pledgerio.app.domain.model.TransactionExtractionDraft import com.pledgerio.app.ui.components.PledgerCard import com.pledgerio.app.ui.components.PledgerTopBar import com.pledgerio.app.util.formatCurrency +import kotlin.math.roundToInt @Composable fun InvoiceScanScreen( onNavigateBack: () -> Unit, onUseDraft: (TransactionExtractionDraft) -> Unit, + onManualEntry: () -> Unit, viewModel: InvoiceScanViewModel = hiltViewModel(), ) { val uiState by viewModel.uiState.collectAsState() + val scanProgress = when (uiState.stage) { + InvoiceScanStage.IDLE -> 0f + InvoiceScanStage.READING_TEXT -> 0.4f + InvoiceScanStage.EXTRACTING_TRANSACTION -> 0.8f + } + val canExtract = uiState.extractedText.isNotBlank() && !uiState.isWorking val imagePickerLauncher = rememberLauncherForActivityResult( contract = ActivityResultContracts.GetContent(), ) { uri -> @@ -66,6 +93,39 @@ fun InvoiceScanScreen( }, ) }, + bottomBar = { + if (uiState.draft != null) { + PledgerCard( + modifier = Modifier.padding( + start = 16.dp, + end = 16.dp, + bottom = 8.dp, + ), + ) { + Text( + text = stringResource(R.string.invoice_scan_ready_title), + style = MaterialTheme.typography.titleMedium, + fontWeight = FontWeight.SemiBold, + ) + Spacer(modifier = Modifier.height(10.dp)) + Button( + onClick = { onUseDraft(uiState.draft) }, + modifier = Modifier + .fillMaxWidth() + .defaultMinSize(minHeight = 48.dp), + ) { + Text(stringResource(R.string.invoice_scan_use_draft_button)) + } + Spacer(modifier = Modifier.height(8.dp)) + OutlinedButton( + onClick = onManualEntry, + modifier = Modifier.fillMaxWidth(), + ) { + Text(stringResource(R.string.invoice_scan_manual_entry)) + } + } + } + }, ) { paddingValues -> Column( modifier = Modifier @@ -77,86 +137,161 @@ fun InvoiceScanScreen( ) { Spacer(modifier = Modifier.height(4.dp)) - Button( - onClick = { imagePickerLauncher.launch("image/*") }, - modifier = Modifier.fillMaxWidth(), - enabled = !uiState.isWorking, - ) { - Icon( - imageVector = Icons.Default.Image, - contentDescription = null, - ) + PledgerCard { Text( - text = stringResource(R.string.invoice_scan_pick_image), - modifier = Modifier.padding(start = 8.dp), + text = stringResource(R.string.invoice_scan_steps_title), + style = MaterialTheme.typography.titleMedium, + fontWeight = FontWeight.SemiBold, ) + Spacer(modifier = Modifier.height(8.dp)) + StepPillRow( + currentStage = uiState.stage, + hasSelectedImage = uiState.selectedImageUri != null, + hasDraft = uiState.draft != null, + ) + if (uiState.isWorking) { + Spacer(modifier = Modifier.height(12.dp)) + LinearProgressIndicator( + progress = { scanProgress }, + modifier = Modifier.fillMaxWidth(), + ) + Spacer(modifier = Modifier.height(6.dp)) + Text( + text = when (uiState.stage) { + InvoiceScanStage.READING_TEXT -> + stringResource(R.string.invoice_scan_reading_text) + InvoiceScanStage.EXTRACTING_TRANSACTION -> + stringResource(R.string.invoice_scan_extracting_transaction) + InvoiceScanStage.IDLE -> "" + }, + style = MaterialTheme.typography.bodySmall, + color = MaterialTheme.colorScheme.onSurfaceVariant, + ) + } } - if (uiState.isWorking) { - LinearProgressIndicator(modifier = Modifier.fillMaxWidth()) + PledgerCard { + Text( + text = stringResource(R.string.invoice_scan_import_title), + style = MaterialTheme.typography.titleMedium, + fontWeight = FontWeight.SemiBold, + ) + Spacer(modifier = Modifier.height(4.dp)) Text( - text = when (uiState.stage) { - InvoiceScanStage.READING_TEXT -> stringResource(R.string.invoice_scan_reading_text) - InvoiceScanStage.EXTRACTING_TRANSACTION -> stringResource(R.string.invoice_scan_extracting_transaction) - InvoiceScanStage.IDLE -> "" - }, + text = stringResource(R.string.invoice_scan_import_hint), style = MaterialTheme.typography.bodySmall, color = MaterialTheme.colorScheme.onSurfaceVariant, ) + Spacer(modifier = Modifier.height(12.dp)) + Button( + onClick = { imagePickerLauncher.launch("image/*") }, + modifier = Modifier + .fillMaxWidth() + .defaultMinSize(minHeight = 48.dp), + enabled = !uiState.isWorking, + ) { + Icon( + imageVector = Icons.Default.Image, + contentDescription = null, + ) + Text( + text = stringResource(R.string.invoice_scan_pick_image), + modifier = Modifier.padding(start = 8.dp), + ) + } } uiState.selectedImageUri?.let { uri -> PledgerCard { + Text( + text = stringResource(R.string.invoice_scan_preview_image_title), + style = MaterialTheme.typography.titleSmall, + fontWeight = FontWeight.SemiBold, + ) + Spacer(modifier = Modifier.height(8.dp)) AsyncImage( model = uri, contentDescription = stringResource(R.string.invoice_scan_image_preview), contentScale = ContentScale.Crop, modifier = Modifier .fillMaxWidth() + .clip(RoundedCornerShape(12.dp)) .height(180.dp), ) } } - OutlinedTextField( - value = uiState.extractedText, - onValueChange = viewModel::onTextChanged, - label = { Text(stringResource(R.string.invoice_scan_text_label)) }, - placeholder = { Text(stringResource(R.string.invoice_scan_text_placeholder)) }, - minLines = 4, - maxLines = 10, - modifier = Modifier.fillMaxWidth(), - ) - - Row( - horizontalArrangement = Arrangement.spacedBy(12.dp), - modifier = Modifier.fillMaxWidth(), - ) { + PledgerCard { + Text( + text = stringResource(R.string.invoice_scan_text_title), + style = MaterialTheme.typography.titleMedium, + fontWeight = FontWeight.SemiBold, + ) + Spacer(modifier = Modifier.height(4.dp)) + Text( + text = stringResource(R.string.invoice_scan_text_hint), + style = MaterialTheme.typography.bodySmall, + color = MaterialTheme.colorScheme.onSurfaceVariant, + ) + Spacer(modifier = Modifier.height(12.dp)) + OutlinedTextField( + value = uiState.extractedText, + onValueChange = viewModel::onTextChanged, + label = { Text(stringResource(R.string.invoice_scan_text_label)) }, + placeholder = { Text(stringResource(R.string.invoice_scan_text_placeholder)) }, + minLines = 4, + maxLines = 10, + modifier = Modifier.fillMaxWidth(), + ) + Spacer(modifier = Modifier.height(12.dp)) Button( onClick = viewModel::extractFromCurrentText, - enabled = !uiState.isWorking, - modifier = Modifier.weight(1f), + enabled = canExtract, + modifier = Modifier.fillMaxWidth(), ) { - Text(stringResource(R.string.invoice_scan_extract_button)) + Icon(imageVector = Icons.Default.AutoAwesome, contentDescription = null) + Text( + text = stringResource(R.string.invoice_scan_extract_button), + modifier = Modifier.padding(start = 8.dp), + ) } } uiState.error?.takeIf { it.isNotBlank() }?.let { message -> - Text( - text = message, - style = MaterialTheme.typography.bodyMedium, - color = MaterialTheme.colorScheme.error, - ) + PledgerCard( + modifier = Modifier + .border( + width = 1.dp, + color = MaterialTheme.colorScheme.error.copy(alpha = 0.35f), + shape = RoundedCornerShape(16.dp), + ), + ) { + Text( + text = stringResource(R.string.error_title), + style = MaterialTheme.typography.titleSmall, + color = MaterialTheme.colorScheme.error, + fontWeight = FontWeight.SemiBold, + ) + Spacer(modifier = Modifier.height(4.dp)) + Text( + text = message, + style = MaterialTheme.typography.bodyMedium, + color = MaterialTheme.colorScheme.onSurface, + ) + } } uiState.draft?.let { draft -> DraftPreviewCard( draft = draft, - onUseDraft = { onUseDraft(draft) }, ) } - Spacer(modifier = Modifier.height(16.dp)) + Spacer( + modifier = Modifier.height( + WindowInsets.navigationBars.asPaddingValues().calculateBottomPadding() + 16.dp, + ), + ) } } } @@ -164,57 +299,183 @@ fun InvoiceScanScreen( @Composable private fun DraftPreviewCard( draft: TransactionExtractionDraft, - onUseDraft: () -> Unit, ) { + val confidence = draft.confidence?.coerceIn(0.0, 1.0) + val confidencePercent = confidence?.times(100)?.roundToInt() + val confidenceColor = when { + confidence == null -> MaterialTheme.colorScheme.onSurfaceVariant + confidence >= 0.85 -> MaterialTheme.colorScheme.primary + confidence >= 0.60 -> MaterialTheme.colorScheme.tertiary + else -> MaterialTheme.colorScheme.error + } + val confidenceLabel = when { + confidence == null -> stringResource(R.string.settings_not_configured) + confidence >= 0.85 -> stringResource(R.string.invoice_scan_confidence_high, confidencePercent ?: 0) + confidence >= 0.60 -> stringResource(R.string.invoice_scan_confidence_medium, confidencePercent ?: 0) + else -> stringResource(R.string.invoice_scan_confidence_low, confidencePercent ?: 0) + } + PledgerCard { - Text( - text = stringResource(R.string.invoice_scan_preview_title), - style = MaterialTheme.typography.titleMedium, + Row( + horizontalArrangement = Arrangement.SpaceBetween, + modifier = Modifier.fillMaxWidth(), ) + { + Text( + text = stringResource(R.string.invoice_scan_preview_title), + style = MaterialTheme.typography.titleMedium, + fontWeight = FontWeight.SemiBold, + ) + ConfidenceBadge( + text = confidenceLabel, + color = confidenceColor, + ) + } Spacer(modifier = Modifier.height(8.dp)) DraftLine( + icon = Icons.Default.Description, label = stringResource(R.string.transaction_description_label), value = draft.description, ) DraftLine( + icon = Icons.Default.Sell, label = stringResource(R.string.transaction_amount_label), value = draft.amount?.formatCurrency(draft.currency ?: "EUR"), ) DraftLine( + icon = Icons.Default.CalendarMonth, label = stringResource(R.string.transaction_date_label), value = draft.date?.toString(), ) DraftLine( + icon = Icons.Default.Person, label = stringResource(R.string.invoice_scan_preview_counterparty), value = draft.targetName ?: draft.sourceName, ) - DraftLine( - label = stringResource(R.string.invoice_scan_preview_confidence), - value = draft.confidence?.let { String.format("%.0f%%", it * 100) }, - ) - Spacer(modifier = Modifier.height(12.dp)) - Button( - onClick = onUseDraft, - modifier = Modifier.fillMaxWidth(), - ) { - Text(stringResource(R.string.invoice_scan_use_draft_button)) + if (draft.description.isNullOrBlank() || draft.amount == null) { + Spacer(modifier = Modifier.height(8.dp)) + Text( + text = stringResource(R.string.invoice_scan_draft_incomplete_hint), + style = MaterialTheme.typography.bodySmall, + color = MaterialTheme.colorScheme.onSurfaceVariant, + ) } } } @Composable private fun DraftLine( + icon: ImageVector, label: String, value: String?, +) { + Row( + horizontalArrangement = Arrangement.spacedBy(10.dp), + modifier = Modifier.fillMaxWidth(), + ) { + Icon( + imageVector = icon, + contentDescription = null, + tint = MaterialTheme.colorScheme.onSurfaceVariant, + modifier = Modifier + .size(18.dp) + .padding(top = 2.dp), + ) + Column(modifier = Modifier.weight(1f)) { + Text( + text = label, + style = MaterialTheme.typography.labelMedium, + color = MaterialTheme.colorScheme.onSurfaceVariant, + ) + Text( + text = value?.takeIf { it.isNotBlank() } ?: stringResource(R.string.settings_not_configured), + style = MaterialTheme.typography.bodyMedium, + ) + } + } + Spacer(modifier = Modifier.height(8.dp)) +} + +@Composable +private fun ConfidenceBadge( + text: String, + color: Color, ) { Text( - text = label, - style = MaterialTheme.typography.labelMedium, - color = MaterialTheme.colorScheme.onSurfaceVariant, - ) - Text( - text = value?.takeIf { it.isNotBlank() } ?: stringResource(R.string.settings_not_configured), - style = MaterialTheme.typography.bodyMedium, + text = text, + style = MaterialTheme.typography.labelSmall, + color = color, + fontWeight = FontWeight.SemiBold, + modifier = Modifier + .clip(CircleShape) + .background(color.copy(alpha = 0.16f)) + .padding(horizontal = 10.dp, vertical = 6.dp), ) - Spacer(modifier = Modifier.height(6.dp)) +} + +@Composable +private fun StepPillRow( + currentStage: InvoiceScanStage, + hasSelectedImage: Boolean, + hasDraft: Boolean, +) { + Row( + horizontalArrangement = Arrangement.spacedBy(8.dp), + modifier = Modifier.fillMaxWidth(), + ) { + StepPill( + label = stringResource(R.string.invoice_scan_step_import), + isActive = currentStage == InvoiceScanStage.IDLE && !hasSelectedImage, + isCompleted = hasSelectedImage, + modifier = Modifier.weight(1f), + ) + StepPill( + label = stringResource(R.string.invoice_scan_step_extract), + isActive = currentStage == InvoiceScanStage.READING_TEXT || + currentStage == InvoiceScanStage.EXTRACTING_TRANSACTION || + (hasSelectedImage && !hasDraft), + isCompleted = hasDraft, + modifier = Modifier.weight(1f), + ) + StepPill( + label = stringResource(R.string.invoice_scan_step_review), + isActive = hasDraft, + isCompleted = hasDraft, + modifier = Modifier.weight(1f), + ) + } +} + +@Composable +private fun StepPill( + label: String, + isActive: Boolean, + isCompleted: Boolean, + modifier: Modifier = Modifier, +) { + val tone = when { + isCompleted -> MaterialTheme.colorScheme.primary + isActive -> MaterialTheme.colorScheme.tertiary + else -> MaterialTheme.colorScheme.onSurfaceVariant + } + val backgroundAlpha = when { + isCompleted -> 0.18f + isActive -> 0.18f + else -> 0.08f + } + + Row( + modifier = modifier + .clip(RoundedCornerShape(999.dp)) + .background(tone.copy(alpha = backgroundAlpha)) + .padding(horizontal = 10.dp, vertical = 8.dp), + horizontalArrangement = Arrangement.Center, + ) { + Text( + text = label, + style = MaterialTheme.typography.labelSmall, + color = tone, + fontWeight = if (isActive || isCompleted) FontWeight.SemiBold else FontWeight.Medium, + ) + } } diff --git a/app/src/main/res/values/strings.xml b/app/src/main/res/values/strings.xml index 52ed1d6..1e3da28 100644 --- a/app/src/main/res/values/strings.xml +++ b/app/src/main/res/values/strings.xml @@ -409,16 +409,31 @@ New template Scan invoice or bill Import an image, extract text, and review the draft + 3-step flow + 1. Import + 2. Extract + 3. Review + Step 1 — Import invoice image + Choose a clear photo or screenshot of your bill. Pick invoice image + Image preview Reading text from image… Extracting transaction details… Selected invoice image + Step 2 — Review extracted text + Fix OCR mistakes before extracting transaction details. Extracted text The invoice text will appear here after OCR. Extract transaction draft Draft preview Counterparty Confidence + High confidence · %1$d%% + Medium confidence · %1$d%% + Low confidence · %1$d%% + Step 3 — Ready to continue + Continue with manual entry + Some values are missing. You can continue and complete them in the transaction form. Use draft in transaction form No results Clear From bd4eadca82eb030de0ee046df7a2cf90f8f176e6 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Mon, 18 May 2026 17:47:15 +0000 Subject: [PATCH 5/8] Fix scan draft smart-cast and localize scan copy Co-authored-by: Gerben Jongerius --- .../app/data/ocr/InvoiceTextExtractor.kt | 5 +-- .../ui/transactions/scan/InvoiceScanScreen.kt | 5 +-- .../transactions/scan/InvoiceScanViewModel.kt | 10 ++++-- app/src/main/res/values-de/strings.xml | 34 +++++++++++++++++++ app/src/main/res/values-nl/strings.xml | 34 +++++++++++++++++++ app/src/main/res/values/strings.xml | 4 +++ 6 files changed, 86 insertions(+), 6 deletions(-) diff --git a/app/src/main/java/com/pledgerio/app/data/ocr/InvoiceTextExtractor.kt b/app/src/main/java/com/pledgerio/app/data/ocr/InvoiceTextExtractor.kt index 9e03fb7..cc6e7e0 100644 --- a/app/src/main/java/com/pledgerio/app/data/ocr/InvoiceTextExtractor.kt +++ b/app/src/main/java/com/pledgerio/app/data/ocr/InvoiceTextExtractor.kt @@ -6,6 +6,7 @@ import com.google.android.gms.tasks.Task import com.google.mlkit.vision.common.InputImage import com.google.mlkit.vision.text.TextRecognition import com.google.mlkit.vision.text.latin.TextRecognizerOptions +import com.pledgerio.app.R import com.pledgerio.app.util.Resource import dagger.hilt.android.qualifiers.ApplicationContext import javax.inject.Inject @@ -26,12 +27,12 @@ class InvoiceTextExtractor @Inject constructor( val text = result.text.trim() recognizer.close() if (text.isBlank()) { - Resource.Error("No readable text found. Try a clearer photo.") + Resource.Error(context.getString(R.string.invoice_scan_error_no_readable_text)) } else { Resource.Success(text) } } catch (e: Exception) { - Resource.Error(e.message ?: "Could not read text from the selected image") + Resource.Error(context.getString(R.string.invoice_scan_error_read_failed)) } } } diff --git a/app/src/main/java/com/pledgerio/app/ui/transactions/scan/InvoiceScanScreen.kt b/app/src/main/java/com/pledgerio/app/ui/transactions/scan/InvoiceScanScreen.kt index 0a8c914..7eacbd4 100644 --- a/app/src/main/java/com/pledgerio/app/ui/transactions/scan/InvoiceScanScreen.kt +++ b/app/src/main/java/com/pledgerio/app/ui/transactions/scan/InvoiceScanScreen.kt @@ -66,6 +66,7 @@ fun InvoiceScanScreen( viewModel: InvoiceScanViewModel = hiltViewModel(), ) { val uiState by viewModel.uiState.collectAsState() + val extractedDraft = uiState.draft val scanProgress = when (uiState.stage) { InvoiceScanStage.IDLE -> 0f InvoiceScanStage.READING_TEXT -> 0.4f @@ -94,7 +95,7 @@ fun InvoiceScanScreen( ) }, bottomBar = { - if (uiState.draft != null) { + if (extractedDraft != null) { PledgerCard( modifier = Modifier.padding( start = 16.dp, @@ -109,7 +110,7 @@ fun InvoiceScanScreen( ) Spacer(modifier = Modifier.height(10.dp)) Button( - onClick = { onUseDraft(uiState.draft) }, + onClick = { onUseDraft(extractedDraft) }, modifier = Modifier .fillMaxWidth() .defaultMinSize(minHeight = 48.dp), diff --git a/app/src/main/java/com/pledgerio/app/ui/transactions/scan/InvoiceScanViewModel.kt b/app/src/main/java/com/pledgerio/app/ui/transactions/scan/InvoiceScanViewModel.kt index c58d55c..b981b43 100644 --- a/app/src/main/java/com/pledgerio/app/ui/transactions/scan/InvoiceScanViewModel.kt +++ b/app/src/main/java/com/pledgerio/app/ui/transactions/scan/InvoiceScanViewModel.kt @@ -1,12 +1,15 @@ package com.pledgerio.app.ui.transactions.scan +import android.content.Context import android.net.Uri import androidx.lifecycle.ViewModel import androidx.lifecycle.viewModelScope +import com.pledgerio.app.R import com.pledgerio.app.data.ocr.InvoiceTextExtractor import com.pledgerio.app.domain.model.TransactionExtractionDraft import com.pledgerio.app.domain.repository.TransactionRepository import com.pledgerio.app.util.Resource +import dagger.hilt.android.qualifiers.ApplicationContext import dagger.hilt.android.lifecycle.HiltViewModel import javax.inject.Inject import kotlinx.coroutines.flow.MutableStateFlow @@ -34,6 +37,7 @@ data class InvoiceScanUiState( @HiltViewModel class InvoiceScanViewModel @Inject constructor( + @ApplicationContext private val context: Context, private val invoiceTextExtractor: InvoiceTextExtractor, private val transactionRepository: TransactionRepository, ) : ViewModel() { @@ -89,7 +93,9 @@ class InvoiceScanViewModel @Inject constructor( fun extractFromCurrentText() { val text = _uiState.value.extractedText.trim() if (text.isBlank()) { - _uiState.update { it.copy(error = "No text to extract. Import an invoice photo first.") } + _uiState.update { + it.copy(error = context.getString(R.string.invoice_scan_error_no_text)) + } return } viewModelScope.launch { @@ -114,7 +120,7 @@ class InvoiceScanViewModel @Inject constructor( _uiState.update { it.copy( stage = InvoiceScanStage.IDLE, - error = result.message, + error = context.getString(R.string.invoice_scan_error_extract_failed), ) } } diff --git a/app/src/main/res/values-de/strings.xml b/app/src/main/res/values-de/strings.xml index 7dd8162..7fca4c6 100644 --- a/app/src/main/res/values-de/strings.xml +++ b/app/src/main/res/values-de/strings.xml @@ -175,6 +175,8 @@ Ausgaben Neue Transaktion Einnahme, Ausgabe oder Überweisung erfassen + Rechnung/Beleg scannen + Text extrahieren und Transaktion vorbefüllen Neues Konto Bankkonto oder Wallet hinzufügen @@ -404,6 +406,38 @@ Keine Konten verfügbar Keine Treffer Neue Vorlage + Rechnung oder Beleg scannen + Bild importieren, Text extrahieren und Entwurf prüfen + 3-Schritte-Ablauf + 1. Import + 2. Extrahieren + 3. Prüfen + Schritt 1 — Rechnungsbild importieren + Wähle ein klares Foto oder einen Screenshot deines Belegs. + Rechnungsbild auswählen + Bildvorschau + Text aus Bild wird gelesen… + Transaktionsdetails werden extrahiert… + Ausgewähltes Rechnungsbild + Schritt 2 — Extrahierten Text prüfen + Korrigiere OCR-Fehler, bevor du Transaktionsdetails extrahierst. + Extrahierter Text + Der Rechnungstext erscheint hier nach der OCR. + Transaktionsentwurf extrahieren + Entwurfsvorschau + Gegenpartei + Sicherheit + Hohe Sicherheit · %1$d%% + Mittlere Sicherheit · %1$d%% + Niedrige Sicherheit · %1$d%% + Schritt 3 — Bereit zum Fortfahren + Mit manueller Eingabe fortfahren + Einige Werte fehlen. Du kannst fortfahren und sie im Transaktionsformular ergänzen. + Kein Text zum Extrahieren. Importiere zuerst ein Rechnungsfoto. + Kein lesbarer Text gefunden. Versuche ein klareres Foto. + Text aus dem ausgewählten Bild konnte nicht gelesen werden. + Transaktionsdetails konnten aus diesem Text nicht extrahiert werden. Bitte prüfe den Text und versuche es erneut. + Entwurf im Transaktionsformular verwenden Keine Ergebnisse Löschen diff --git a/app/src/main/res/values-nl/strings.xml b/app/src/main/res/values-nl/strings.xml index 62cd2aa..73ec6b8 100644 --- a/app/src/main/res/values-nl/strings.xml +++ b/app/src/main/res/values-nl/strings.xml @@ -175,6 +175,8 @@ Uitgaven Nieuwe transactie Inkomsten, uitgaven of overboeking registreren + Factuur/rekening scannen + Tekst ophalen en transactie vooraf invullen Nieuwe rekening Bankrekening of portemonnee toevoegen @@ -404,6 +406,38 @@ Geen rekeningen beschikbaar Geen resultaten Nieuw sjabloon + Factuur of rekening scannen + Importeer een afbeelding, haal tekst op en controleer het concept + Stappenplan in 3 stappen + 1. Importeren + 2. Uitlezen + 3. Controleren + Stap 1 — Factuurafbeelding importeren + Kies een duidelijke foto of screenshot van je rekening. + Factuurafbeelding kiezen + Voorbeeldafbeelding + Tekst uit afbeelding lezen… + Transactiegegevens extraheren… + Geselecteerde factuurafbeelding + Stap 2 — Opgehaalde tekst controleren + Corrigeer OCR-fouten voordat je transactiegegevens extraheert. + Opgehaalde tekst + De factuurtekst verschijnt hier na OCR. + Transactieconcept extraheren + Conceptvoorbeeld + Tegenpartij + Betrouwbaarheid + Hoge betrouwbaarheid · %1$d%% + Gemiddelde betrouwbaarheid · %1$d%% + Lage betrouwbaarheid · %1$d%% + Stap 3 — Klaar om door te gaan + Doorgaan met handmatige invoer + Sommige waarden ontbreken. Je kunt doorgaan en ze invullen in het transactiescherm. + Geen tekst om te extraheren. Importeer eerst een factuurfoto. + Geen leesbare tekst gevonden. Probeer een duidelijkere foto. + Kon geen tekst lezen uit de geselecteerde afbeelding. + Kon geen transactiegegevens uit deze tekst halen. Controleer en probeer opnieuw. + Concept gebruiken in transactiescherm Geen resultaten Wissen diff --git a/app/src/main/res/values/strings.xml b/app/src/main/res/values/strings.xml index 1e3da28..024f9e3 100644 --- a/app/src/main/res/values/strings.xml +++ b/app/src/main/res/values/strings.xml @@ -434,6 +434,10 @@ Step 3 — Ready to continue Continue with manual entry Some values are missing. You can continue and complete them in the transaction form. + No text to extract. Import an invoice photo first. + No readable text found. Try a clearer photo. + Could not read text from the selected image. + Couldn\'t extract transaction details from this text. Please review and try again. Use draft in transaction form No results Clear From 19b5684ad8a1d7d0d6eafe5b4cf639a20ab53ec5 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Mon, 18 May 2026 17:57:25 +0000 Subject: [PATCH 6/8] Add camera capture support for invoice scanning Co-authored-by: Gerben Jongerius --- app/src/main/AndroidManifest.xml | 10 +++ .../ui/transactions/scan/InvoiceScanScreen.kt | 63 +++++++++++++++++++ .../transactions/scan/InvoiceScanViewModel.kt | 6 ++ app/src/main/res/values-de/strings.xml | 2 + app/src/main/res/values-nl/strings.xml | 2 + app/src/main/res/values/strings.xml | 2 + app/src/main/res/xml/file_paths.xml | 6 ++ 7 files changed, 91 insertions(+) create mode 100644 app/src/main/res/xml/file_paths.xml diff --git a/app/src/main/AndroidManifest.xml b/app/src/main/AndroidManifest.xml index 44962ef..3fd6433 100644 --- a/app/src/main/AndroidManifest.xml +++ b/app/src/main/AndroidManifest.xml @@ -38,6 +38,16 @@ + + + +