Skip to main content

Synchronization

The synchronization system keeps client data in sync with the PHP backend through a long-poll mechanism. It is implemented in libs/upp-data/src/modules/sync.ts and coordinates with the server's connection/sync endpoints.

Architecture Overview​

Key Classes​

syncService​

The Angular injectable (providedIn: 'root') that orchestrates everything. Key responsibilities:

  • Session lifecycle: Start(session) / Stop()
  • Entity association: SetUser(user), SetPlace(place), SetQrCode(qrcode)
  • Change flushing: Flush(changes, force) sends committed changes to the server
  • Observables: OnChange, OnExpired, OnReleased, IsReady
@Injectable({ providedIn: 'root' })
export class syncService implements OnDestroy {
    constructor(
        public lang: languageService,
        public store: storeService,
        public cache: cacheService,
        public adhoc: adhocService,
        public state: stateService,
        public clock: clockService,
        public toast: toastService,
        private deviceid: identificationService,
        private alertCtrl: alertService
    ) { ... }
}

Connection​

Manages the server connection lifecycle and periodic refresh:

MethodPurpose
CreateConnection()Sends connection/create with triggers, then starts periodic refresh
RefreshConnection()Re-sends current triggers to connection/update
ReleaseConnection()Sends connection/delete, runs final sync, stops refresh
SetUser(user)Updates triggers with user info
SetPlace(place)Updates triggers with place info
SetQrScan(qrscan)Updates triggers with QRSCAN + PLACE (guest mesa wake)
SetQrCode(qrcode)Refreshes QRSCAN/PLACE from the linked scan
CancelRefresh()Asks the server (connection/cancel) to end the long-poll; the synchronizer also aborts the in-flight client /sync request

Connection status: RUNNING | STOPPED | EXPIRED.

Syncronizator​

The core sync engine. Manages:

  • Abortable long-poll: each /sync arms a cancel handle on adhocService.DoRequest. CancelRefresh (flush, wake, or watchdog) aborts the client XHR first so _onrefresh cannot stick after OS suspend; intentional abort requeues sent packages without transport backoff.
  • Wake / watchdog recovery (StaleRefreshGuard): on visibilitychange → visible (or pageshow with bfcache), or when a refresh stays in flight beyond ~45 s, cancel if needed, promote wait→sync, and force a fresh refresh (deduped ~3 s).
  • Change queues: _syncchanges (pending send), _waitchanges (staging while a refresh is in flight), _sentchanges (in-flight). _waitchanges is promoted into _syncchanges on every DoRefresh exit — including early skips when not connected or when the client is offline (!IsOnline) — so packages are never stranded in wait. While offline the cycle does not call /sync (quiet skip, up to 60 s). When connectivity returns, that wait ends and the same cycle sends the pending queue. If reconnect finds a stuck in-flight /sync (no delay armed), the client aborts it and forces a fresh cycle so the queue is not blocked until the HTTP timeout.
  • In-transit FK requires: a local flush may reference a parent that is still in _sentchanges, already in ResolvedMap, or mid-commit in the live store (InCommit, same rule as data.Commit) without raising relation_not_found. The child is staged locally. The pending sync queue is sent atomically: the whole package goes only when every open require has an objid or its parent travels in the same package; otherwise the cycle sends nothing. If application retries are exhausted, the entire pending set (sync / sent / wait) is discarded.
  • Timestamps: Per-target (SESSION, PLACE, QRCODE) to enable incremental sync
  • Server reference: Adjusts local timing to server clock
  • Dependency resolution: CheckDepends() ensures changes are ordered correctly
  • UUID → objid mapping: ResolvedMap tracks server-assigned IDs for locally-created objects

PendingQueue​

Manages incoming server updates with priority-based processing:

  • Updates are assigned numeric priorities based on their table type
  • Lower priority numbers are processed first (synchronously when possible)
  • Processing is batched (up to 5000 items for high-priority, 500 for low) with a 50ms time limit per batch
  • Progress tracking via OnRefreshStage and OnRefreshPercent

CacheManager​

Persists sync data to IndexedDB for faster startup:

  • Session cache: Stores session-level updates keyed by (deviceid, sessionid)
  • Place cache: Stores place-level updates keyed by placeid
  • Validates cache freshness against AppConstants.CacheExpiration using client wall-clock milliseconds (Date.now()), not the server sync watermark (Unix seconds)
  • On startup, loads cached data into PendingQueue before querying the server

Update Triggers​

When a connection is created or updated, the client sends triggers that tell the server what data this connection cares about:

interface UpdateTriggers {
    SESSION: string;      // always required
    QRSCAN: string | null; // guest mesa wake (global scan id, unique across places)
    USER: string | null;
    PLACE: string | null;
}

The server stores each connection's triggers in connection state (Redis / APCu / CONNECTED, per DATA_SHARE_MODE). When changes are committed that affect matching triggers, the server marks those connections for refresh. Guest QRA registers PLACE + QRSCAN; ticket updates propagate to QRCODE and the server adds a notify-only QRSCAN descriptor so the guest wakes without relying on tenant-local QRCODE.objid values (which can collide across place databases).

Sync Flow​

1. Starting a Session​

await syncService.Start(sessionId);

This:

  1. Sets state.session
  2. Recovers any pending changes from previous sessions (via storeService)
  3. Loads cached session data from IndexedDB
  4. Creates a server connection with triggers
  5. Begins periodic refresh

2. Periodic Refresh Cycle​

Every AppConstants.MainRefresh milliseconds:

  1. Prepare changes: Collects uncommitted changes, resolves UUID→objid for previously inserted items, assigns server time reference
  2. Send request: POST /sync with { access, timestamp, target } as query params and changes as body
  3. Process response:
    • Store newly resolved UUID→objid mappings
    • Save updates to IndexedDB cache
    • Filter out volatile responses and already-pending items
    • Enqueue updates into PendingQueue for prioritized processing
  4. Handle expiration: If response.expired, reconnect automatically
  5. Handle errors: Retry with backoff (2s for HTTP errors, 10s for server errors)

3. Committing Changes​

When the client modifies data:

// DataObject collects changes recursively
const changes = ticket.Commit;

// Queue changes for next sync cycle
await dataService.Commit(changes);
await dataService.Flush(force);

The Syncronizator.FlushChanges() method:

  1. Deduplicates changes (same object replaces previous pending change)
  2. Validates dependency ordering via CheckChanges()
  3. Persists changes to local storage as a safety net
  4. If force = true, cancels any running refresh and triggers an immediate one

4. Conflict Resolution​

The system uses a last-write-wins strategy with merge:

private _MergeChange(chng1: any, chng2: any): any {
    const updated1 = new Date(chng1['updated']);
    const updated2 = new Date(chng2['updated']);
    const source = updated1 > updated2 ? chng1 : chng2;
    const target = updated1 > updated2 ? chng2 : chng1;
    return { ...target, ...source };
}

When the same object has both a pending local change and a server update:

  • The entry with the newer updated timestamp takes precedence
  • Fields from both are merged, with the newer version's fields overriding

5. Dependency Ordering​

Before sending changes, CheckChanges() ensures correct ordering:

  1. Topological sort: If change A depends on change B (references B's UUID), B must come first
  2. Relation map validation: Every UUID reference must either already have an objid or be present earlier in the change list
  3. Throws a descriptive error if ordering cannot be satisfied

Priority System​

The PendingQueue assigns priorities to incoming updates based on table type. Lower numbers are processed first:

PriorityTablesRationale
1.1–1.5SESSION, FCM, USER, STAFF, PLACECritical session/identity data
2.1–2.7ADDRESS, STRIPE, PLACEOPT, AUDIT, etc.Infrastructure without UI components
3.1–3.8PRODUCT, CATEGORY, OFFER, EXTRA, DISCOUNT, etc.Catalog data (needed before tickets)
4.1–4.2FAMILY, FAMILYPRODUCT, FAMILYPERIODProduct groupings
5.1–5.6TICKET*, SESSIONActive tickets and their components
6.1–6.3QRCODE, PRINTER, PAYMENT, PLACEPlace configuration updates
8.1–8.6(same tables, lower status)Deferred items (e.g., deleted QR codes)

Items with _actn == 'do_insert' get a -10 offset, ensuring inserts are processed before updates.

Stages 1–3 are processed synchronously (blocking the UI clock), while stages 4+ are processed asynchronously in 50ms batches with yields to the event loop.

Server-Side Sync​

On the PHP backend, the sync flow (model/synchronize.php) works as follows:

  1. Validate session and commit POST body changes in a DB transaction
  2. Call connection_notify to mark affected connections as REFRESH
  3. Long-poll: If timestamp > 0 and status is UPDATED, poll in a loop (~20s) waiting for REFRESH, EXPIRED, or CANCELLED
  4. When refresh is due, load updates from the correct database:
    • Central DB for user-target data (SESSION, USER, PLACE, STAFF)
    • Tenant DB for place-target data (PRODUCT, TICKET, QRCODE, etc.), including _CHANGESLOG payloads whose table may be a global graph row (SESSION, QRSCAN, STAFF, …) projected onto that tenant so connected POS devices see guest/staff session state without polling

Place-graph projection (global rows → tenant changelog)​

SESSION, QRSCAN, STAFF, USER (when linked to staff), FCM, and POSDEVICE live in the central database. Writers outside a sync batch call publish_place_graph_changes after the central commit; each projected batch also includes { table: PLACE, objid } so _check_notify can match POS connections by their PLACE trigger (global graph rows are not children of PLACE in datamodel.json). Rows touched inside synchronize use SyncDelayed::touch() and ride the same stage_changes / notify_changes path as tenant tables (_propagate already adds PLACE for tenant children). There is no _CHANGESLOG on the global schema. Heartbeats (touch_session) are not published, and SESSION.touch is rewritten at most once per minute per session. A sync batch never holds a central and a tenant transaction at the same time: central entries are applied and committed first, the tenant transaction follows, and central writes derived from tenant entries (for example completing a QRSCAN after creating a QRCODE) run after the tenant COMMIT; replaying the batch is idempotent by _uuid. Logout of a table session is release_session plus that projection — not refresh_places / CONNECTED. 5. Return JSON: { errorcode, timestamp, reference, target, updated, expired, updates[] }

Connection State​

Connection state backend is selected by DATA_SHARE_MODE (Deploy::share_mode() / AppConstants::shareMode(), default SAFE):

ModeWhere connection state lives
REDISRedis (Sentinel primary; keys _UPPCONN_ + session, TTL 60 s; PUB/SUB place/global channels). Multi-node. Temporary Redis outage → Critical + NotConnected (no silent fallback to SAFE).
APCUAPCu on the node (60 s TTL). Single-node / lab.
SAFECONNECTED table (central DB). Column session is INT → SESSION.objid (schema relation). Callers still pass SESSION.id; SAFE resolves to objid before read/write. Redis/APCu keep the string key.
  • All reads/writes of connection state (create, refresh, update, delete, cancel, notify) run under an exclusive mutex: with_lock('connection_state', MUTEX_NAME, …) from legacy/base/mutex.php. The mutex backend follows the same shareMode() (MutexRedis vs local flock) unless a caller forces MUTEX_LOCAL. Factory details: server contract legacy/base/mutex.mdc.
  • Fields: session, status (UPDATED/REFRESH/CANCELLED), synchronized, config (JSON triggers)

Note: Fiscal registry appends use a separate lock (MUTEX_FILE on the shared registry path), not connection_state. Front-end GlobalMutex / storeService locks are unrelated.

Triggers and Notification​

Each connection's config holds triggers mapping tables to object IDs. When changes are committed:

  1. The server builds a list of (table, objid) updates
  2. connection_notify matches these against all active connections' triggers
  3. Matching connections are marked REFRESH
  4. Only those clients receive data on their next poll

Session Expiration​

When the server returns errorcode == 2 or errorcode == 3:

  1. syncService.OnSessionExpired() is called
  2. The synchronizer status is set to EXPIRED
  3. The connection is stopped
  4. A modal alert notifies the user
  5. OnExpired observable emits, allowing the app to redirect to login

Stored Changes (Persistence)​

Changes awaiting sync are persisted to storeService (Ionic Storage) under the key upp-stored-syncdata. This ensures that if the browser is closed or crashes:

  • On next startup, GetStoredChanges() recovers them
  • They are sent with the next sync cycle
  • After successful sync, DelStoredChanges() clears the stored copy

Retry limits prevent infinite loops: 50 retries for HTTP errors, 5 for server errors. If exhausted, changes are discarded and the user is notified.

Local Backup (Fiscal Devices)​

storeService (browser storage) is the primary store for pending changes, but on local fiscal devices (a POS connected to a Raspberry Pi print/cache service over a local WebSocket) it is not durable enough on its own: if the browser storage is lost while the connection to the server is down, unsent fiscal changes could disappear. wsstoreService (libs/upp-data/src/modules/wsconnect.ts) adds a second, independent copy of the same pending changes on that local service, so they can be recovered even if browser storage fails while offline.

When it activates​

  • Place.RequiresLocalBackup is true only when the active device connection is local (RasppiConnect.IsLocal, i.e. the POS talks to a host:port on the same network, not to the cloud service).
  • When RequiresLocalBackup is true, Syncronizator opens a connection to the local cache through wsstoreService and dual-writes every pending-changes update (Set/Del on upp-stored-syncdata) to both storeService and the local cache.
  • Place.WSStorageReady reflects whether that backup is currently reliable: it is always true when RequiresLocalBackup is false (nothing to guarantee), and otherwise mirrors Place.BackupReady, which Syncronizator flips to true once the local connection is open and any previously backed-up changes have been recovered/merged, and back to false immediately on disconnect.

Blocking ticket sending: AllowTicketCommit​

Place.AllowTicketCommit gates creating/sending new tickets (upp-ticket-edit's "send" action, table/bar ticket creation, ticket thumb "prepare"). It requires the place license to be valid and WSStorageReady. In practice this means: on a local fiscal device, the client refuses to create new tickets until the local backup connection is confirmed reachable, so that any ticket committed afterwards is guaranteed to have a second copy of its pending change. The topbar shows a NOSERVICE warning whenever !WSStorageReady, which is the same condition that blocks sending.

Blocking payment/closing: AllowTicketPayment​

Closing/paying a ticket (cash, card) is gated by the stricter Place.AllowTicketPayment, which additionally requires:

  • Either connectivity to the server (viewService.IsOnline) or a ready local backup (covered by AllowTicketCommit).
  • If TicketBAI (Basque fiscal signing, SendBAIEnabled) is enabled and the device is offline, the signing libraries must already be preloaded (Ticketbai.LibrariesReady) — Syncronizator.SetPlace triggers a best-effort background preload (Ticketbai.PreloadLibraries) as soon as a place with local backup is set, so the libraries are typically ready before the till operator needs to close a ticket offline.

This lets fiscal signing happen offline once the libraries are preloaded, while still preventing a payment from being closed on a local device whose backup is not yet confirmed.

Best-effort guarantee​

The local backup is a best-effort safety net, not a second source of truth: a transient write failure to the local cache never blocks or rolls back the primary storeService write, and the server remains the single canonical source once connectivity is restored (see Sync Flow above). Local backup only reduces the window in which a pending change could be lost while offline on a fiscal device.