Skip to content

Fix/get folders utf8#6

Open
frederic34 wants to merge 67 commits into
Eoxia:developfrom
frederic34:fix/get-folders-utf8
Open

Fix/get folders utf8#6
frederic34 wants to merge 67 commits into
Eoxia:developfrom
frederic34:fix/get-folders-utf8

Conversation

@frederic34

Copy link
Copy Markdown

No description provided.

frederic34 and others added 30 commits June 16, 2026 16:19
PHP's imap_utf7_decode() returns raw UTF-16BE bytes instead of valid
UTF-8 for non-ASCII IMAP folder names (e.g. "Envoyés" encoded as
"Envoy&AOk-s"). This caused json_encode() to silently return false,
producing an empty HTTP response body despite a 200 status.

Fix: convert IMAP modified UTF-7 to standard UTF-7 then use
mb_convert_encoding(..., 'UTF-8', 'UTF-7') in getFolders().
Also add json_encode() error checking in get_folders.php to surface
any future encoding failures explicitly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
document.getElementById('email-empty-state') and 'email-view-content'
returned null (elements not in HTML), throwing a TypeError that stopped
execution before fetchEmails() was called. Added null guards so the
click handler reaches fetchEmails() regardless.

Also removed duplicate <script> tag loading app.js at the bottom of
index.php — it was already injected via llxHeader() through $head.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Nouvelle constante INBOX_REFRESH_INTERVAL (secondes, 0 = désactivé)
  configurable dans admin/setup.php > Paramètres globaux
- setInterval() dans app.js recharge fetchEmails() à l'intervalle choisi,
  sauf si le formulaire de réponse est ouvert
- Bouton .fa-sync câblé pour déclencher fetchEmails() manuellement
- inboxRefreshInterval passé en JS depuis index.php via getDolGlobalInt()

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Load emails 50 at a time; IntersectionObserver triggers the next page
when the user reaches the bottom of the list.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ith scroll listener

IntersectionObserver only fires on state changes; if the sentinel remains
visible after a page load the callback never re-triggers. A scroll listener
on the container checks position on every scroll event instead.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ter each page

The observer only fires on state changes (not-visible → visible).
After a page loads, if the sentinel is still visible the state stays
"intersecting" and never transitions again, so no further pages load.
Re-arming (unobserve + observe) after each successful fetch forces
a fresh evaluation; if the sentinel is still visible it fires immediately
and keeps loading until it scrolls off-screen or has_more=false.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
limit_nb was slicing the full result set before pagination, so with
limit_nb=100 and page_size=50 the pool was capped at 100 and the second
page always returned has_more=false even if more emails existed.
With pagination, offset/page_size already controls per-request load;
limit_days provides the time-based scope. Removing the pre-slice lets
total reflect the actual mailbox count.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- IMAPClient::moveMessage() moves a message to a destination folder
- IMAPClient::deleteMessage() permanently deletes as fallback
- ajax/trash_email.php handles the POST request
- app.js tracks trashFolder from the folder list and wires up the
  trash button; removes the item from the list and clears the view panel

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- IMAPClient::getAttachments() walks the MIME tree and returns filename,
  MIME type, size and part number for each attachment
- IMAPClient::getAttachmentData() fetches raw bytes for a given part
- get_email_body.php now also returns attachments[] in the JSON
- ajax/get_attachment.php serves attachment bytes as a file download
- app.js renders a chip bar below the email body for each attachment;
  each chip is a direct download link

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
HTML emails contain full <html>/<head>/<style> documents; injecting them
via innerHTML breaks the page CSS. Using srcdoc on a sandboxed iframe
isolates the email rendering completely. Scripts are blocked (no
allow-scripts), links open new tabs (allow-popups). iframe height is
auto-adjusted after load to fit content.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ffset

imap_expunge() renumbers sequence numbers after deletion, causing all
subsequent msgno-based lookups to fetch the wrong message. UIDs are
permanent and never change after expunge.

- imap_search() now uses SE_UID flag → returns UIDs
- imap_fetch_overview() uses FT_UID flag
- imap_fetchstructure/fetchbody/body use FT_UID
- imap_mail_move uses CP_UID, imap_delete uses FT_UID
- All AJAX endpoints accept 'uid' param instead of 'msgno'
- Attachment metadata now includes 'encoding' field (was missing,
  causing corrupted downloads when encoding != identity)
- JS uses email.uid everywhere

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- llx_inbox_comment table (fk_account + folder + message_uid identify the email)
- InboxComment class: create(), fetchByMessage(), deleteComment() (soft delete)
- ajax/get_comments.php — load comments for an email (with author initials)
- ajax/add_comment.php  — add a comment, returns formatted comment for display
- ajax/delete_comment.php — soft-delete own comment (or any if admin)
- app.js: loads comments when email is opened, renders list with delete button,
  submit on button click or Ctrl+Enter, updates badge count

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
IMAP UIDs are per-folder; moving an email assigns a new UID in the
destination folder. The RFC 2822 Message-ID header is the only
identifier stable across folder moves.

- getMessages() now exposes message_id from imap_fetch_overview
- fetchByMessage() searches by message_id when present, falls back
  to (fk_account, message_uid) for legacy rows without message_id
- folder is kept in the table for audit/info but not used for lookup
- get_comments and add_comment pass message_id through
- JS passes email.message_id to loadComments() and add_comment

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- getMessages() now exposes 'to' and 'cc' from imap_fetch_overview
- email view updates .sender-email with actual À/Cc recipients
- sender avatar now shows initials derived from the From display name

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Toggle button in sidebar header collapses it to icon-only (56px).
State persists in localStorage. Folder items show tooltip in
collapsed mode. Smooth 0.2s CSS transition.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add 'prefix' key to menu entry — 'picto' is only used in the admin
modules list, 'prefix' is what gets stored in llx_menu and rendered
in the top navigation bar.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Parse "Name <email>" format for From, To, Cc fields.
Display names get a dotted underline and title= attribute
with the raw email address. Bare addresses shown as-is.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
IMAPClient: class docblock, @var on all properties, full @param/@return
  on every method including private helpers; corrected $uid vs $msgno
  mislabel in getMessageBody.

InboxAccount: class docblock explaining shared vs personal accounts,
  @var on all 18 properties, tightened @param/@return on create/fetch.

InboxComment: class docblock explaining soft-delete and Message-ID
  strategy, @var on all properties including joined user_* fields,
  full @param/@return on create/fetchByMessage/deleteComment;
  (void) $folder in fetchByMessage to suppress unused-param hint.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- get_emails.php: add NOTOKENRENEWAL/NOREQUIREMENU/NOCSRFCHECK (same
  pattern as all other AJAX endpoints in this module)
- get_folders.php: add missing NOCSRFCHECK define
- modInbox.class.php: use isModEnabled()/hasRights() (Dolibarr 17+ API),
  picto fa-envelope, user=0 (internal only), drop redundant picto key

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Same padding/border-radius/height as the primary button, secondary
variant (transparent bg, border). In collapsed mode both buttons
become 36px circles, matching each other.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add llx_inbox_tag (tag catalog: label, color, imap_keyword) and
  llx_inbox_message_tag (per-message associations keyed on message_id)
- InboxTag and InboxMessageTag classes with full CRUD
- Four AJAX endpoints: get_tags, get_message_tags, add/remove_message_tag
  (IMAP keyword flag synced server-side when imap_keyword is set)
- Admin page admin/tags.php for tag management (color picker + keyword)
- IMAPClient: read keywords from imap_fetch_overview, setKeyword/clearKeyword
- Front-end: tag picker dropdown, chips with remove button, auto-loaded on email open
- modInbox: register new SQL tables via _load_tables, expose tags admin page

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add lib/inbox.lib.php with adminInboxPrepareHead() returning tabs
for the two setup pages (Comptes / Tags). Both admin/setup.php and
admin/tags.php now use dol_get_fiche_head/dol_get_fiche_end to render
the standard Dolibarr tab strip. Lang keys InboxAccounts and InboxTags
added to fr_FR.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
frederic34 and others added 30 commits June 18, 2026 23:27
…exists

Snapshot ordered UIDs before clearing. After rebuild:
- email still present → restore .active silently (no re-open)
- email gone → search forward then backward in old order for nearest
  surviving neighbor and click() it; empty list → hide view panel
…upport

- Add composer.json + composer.lock (bytestream/horde-imap-client v2.34)
- Rewrite IMAPClient using Horde_Imap_Client_Socket (no ext/imap needed)
- Add getSyncToken() and getDelta() for CONDSTORE/QRESYNC delta sync
- Add vendor/ to .gitignore (run composer install to restore)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The Horde sort option triggered a client-side sort (clientSort) that
silently returned 0 results when the server didn't support RFC 5256 SORT.
UIDs are monotonically increasing so rsort() gives newest-first order
without any extra round-trip.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Horde_Imap_Client returns flags as lowercase (\seen, \answered)
while the previous code compared against titlecase (\Seen, \Answered).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
.inbox-panel-content padding (15px 20px) was creating wasted space
on each side of the email list. Items already have their own padding.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…MAP when opening a message

- Add INBOX_BLOCK_REMOTE_IMAGES setting (default: on) in admin/setup.php
- In get_email_body.php, expose block_images flag in JSON response
- In app.js, replace remote src attributes with data-original-src before injecting
  into iframe; show a warning banner with an "Afficher les images" button
- Add markSeen() method to IMAPClient using Horde store() with \Seen flag
- Add ajax/mark_seen.php endpoint
- Call mark_seen.php (fire-and-forget) when clicking an unread message so the
  \Seen flag is persisted to the IMAP server and the read state survives a refresh
- Remove dummy email item from the list (was cleared by JS anyway but caused a flash)
- Hide panel-view on initial load (display:none); JS reveals it on email click
- Empty sender, subject, date and body elements — content is set by JS
Replace display:none/flex toggling on panel-view with a clearViewPanel()
function that empties the content in place. The panel now occupies its
column permanently so the ERP context panel does not shift when no
message is selected or after deletion.
- Add auth_type and oauth_service fields to InboxAccount
- IMAPClient::connect() resolves DoliStorage token and passes xoauth2_token to Horde
- Fix "No password provided" — set placeholder password alongside xoauth2_token to bypass Horde early check
- Admin UI: auth type radio, OAuth2 provider dropdown filtered to IMAP-capable scopes
- SQL migration: llx_inbox_account-alter.sql
- Lang keys for OAuth2 UI (EN + FR)
- Documentation README.md (EN) and README-FR.md (FR)
- Sidebar split into account list + folder list sections
- get_accounts.php: returns all active accounts for current user
- get_unread_counts.php: IMAP INBOX unseen count per account
- All AJAX endpoints now accept account_id parameter
- app.js: fetchAccounts → fetchFolders(accountId) chain, currentAccountId state
- Unread badge (blue, 99+ cap) on each account, refreshed with auto-refresh interval
…NARY extension

Gmail (and most IMAP servers) do not advertise the BINARY extension (RFC 3516),
so Horde's 'decode => true' option in bodyPart() has no effect and returns raw
base64/quoted-printable content. Add transferDecode() helper that reads the
Content-Transfer-Encoding from the MIME structure and decodes accordingly.
Applied to getMessageBody() and getAttachmentData().

Also strip [Gmail]/ namespace prefix from folder display names and map \All
special-use to archive type. Fix placeholder password for Horde XOAUTH2 login.
…ad getPart call

transferDecode() was calling non-existent getTransferEncoding() on Horde_Mime_Part.
Fix: add mimeHeader() to the FETCH query alongside bodyPart(), then parse
Content-Transfer-Encoding with a regex on the raw header string.
Also remove the now-unused structure/getPart calls in getAttachmentData().
- IMAPClient::markUnseen() — removes \Seen flag via IMAP store
- mark_seen.php — accepts seen param (1=read, 0=unread)
- buildEmailEl() — hover overlay with envelope toggle and trash button
- Buttons use stopPropagation so click-to-open still works independently
- Trash removes item from list and clears view panel if it was open
- Both actions refresh the unread badge counter
formatDate(): À l'instant / X min / HH:MM / Hier HH:MM / 15 juin / 15 juin 2024
formatDateFull(): vendredi 20 juin 2026 à 14:32 for the view panel header
Raw date kept in title attribute for exact timestamp on hover
Add InboxProviderInterface defining the contract for all messaging
backends (IMAP, Graph, Gmail, WhatsApp, SMS…). Implement ImapProvider
as a thin adapter over the existing IMAPClient.

Add InboxProviderFactory (switch on account.provider_type) and
add_provider_type column migration. Migrate all 10 AJAX endpoints
to use InboxProviderFactory::create() instead of new IMAPClient()
directly, so swapping providers requires no changes in the AJAX layer.
Add WhatsAppProvider implementing InboxProviderInterface on top of
Meta's Graph API. Messages are stored locally in
llx_inbox_whatsapp_message (populated by ajax/whatsapp_webhook.php).

- WhatsAppProvider: getMessages/getThreadedMessages query local DB;
  markSeen sends a read receipt via Graph API; sendTextMessage posts
  to /{phone_number_id}/messages and stores the copy locally.
- whatsapp_webhook.php: handles Meta's GET verification handshake
  and POST events (text, image, document, audio, video, sticker,
  location, status updates). NOLOGIN endpoint, secured by verify_token.
- inboxaccount: add config TEXT column (JSON for provider settings)
  with getConfig() helper; factory activates the whatsapp case.
- send_email.php: WhatsApp accounts bypass SMTP and call
  WhatsAppProvider::sendTextMessage() directly.
- SQL: create_inbox_whatsapp_message.sql, alter_inbox_account_add_config.sql
Reflect the provider abstraction layer (InboxProviderInterface,
InboxProviderFactory, ImapProvider, WhatsAppProvider), updated
database schema (provider_type, config, llx_inbox_whatsapp_message),
WhatsApp setup guide (Meta app creation, webhook configuration,
SQL insert), and updated file structure.
Add provider_type radio selector (IMAP vs WhatsApp) to the account
form. WhatsApp section shows Phone Number ID, Access Token (password
field), Verify Token, API version, and the computed webhook URL to
paste into Meta Developer Console.

IMAP/Auth/SMTP sections are hidden when WhatsApp is selected via JS.
Account list shows a coloured badge (green = WhatsApp, blue = IMAP).
Test action verifies WhatsApp credentials against the Graph API and
shows the verified phone number on success. Delete action implemented.
New translation keys added (EN + FR).
…of current)

JS: add account_id to FormData in addMessageTag() and removeMessageTag().
PHP: add_message_tag and remove_message_tag now prefer the posted
account_id (with access-control check) and fall back to the first
accessible account only when account_id is absent.
Add getThreadedMessages() to IMAPClient using IMAP THREAD command
(REFERENCES algorithm with ORDEREDSUBJECT fallback). Add thread
list UI styles, conversation card layout, and toggle button in the
message list panel. Fix admin tab URLs with dolBuildurl() wrapper.
get_message_tags.php was ignoring the active account and always
querying with the first accessible account's rowid. Tags saved on
account N were never found when viewing that account.

Fix: accept account_id param and validate access, matching the
pattern already applied to add/remove_message_tag.php. Pass
account_id from JS in loadMessageTags().
- Replace raw SQL insert with admin UI instructions for WhatsApp accounts
- Clarify that webhook URL is shown on the edit form after saving
- Document that tags are global (entity-level) definitions, assignments are per-account
- Add note on IMAP keyword sync compatibility: not supported on Gmail or WhatsApp
Show only Bold, Italic, Underline, Strike, RemoveFormat, lists and
Link — removes the full default CKEditor toolbar which is too heavy
for a reply composer. Also removes the element path bar and disables
manual resize.
Add 23 translation keys to both lang files (JustNow, Yesterday,
MarkRead/Unread, MoveToTrash, NoEmailFound, WroteOn, SendingIn,
RemoveTag, etc.). Inject them from index.php into window.inboxLangs
using transnoentitiesnoconv(). Pass browser locale from
$langs->defaultlang so toLocaleString() formats dates correctly.
Replace all hardcoded French strings in app.js with inboxLangs.*
references.
…troller

- Remove leftover prototype event listeners and console.log calls
- Cancel in-flight IMAP fetch when user switches folder so stale results
  never overwrite the newly selected folder's email list
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant