Project History Sync State Collection

projectHistorySyncState holds one document per project that has been resynced: the state of the current or last resync, when project-history rebuilds a project’s history from what web and docstore hold. It is owned by project-history and has no Mongoose schema; the code is the SyncState class and its helpers in services/project-history/app/js/SyncManager.js.

Fields

Field	Type	Meaning	Source
_id	ObjectId	Created by the upsert.	SyncManager.js:233-237
project_id	ObjectId	The project.	SyncManager.js:233-237
resyncProjectStructure	Boolean	true while project-history waits for the project structure (file tree) from web.	SyncManager.js:457, 536-543
resyncDocContents	Array of String	Paths of the docs whose content it still waits for.	SyncManager.js:458, 482, 545-546
origin	Object	Who asked for the resync, for example { kind: 'history-resync' }, the default.	SyncManager.js:150, 459
hardResync	Boolean	true for a hard resync, which first drops the queued updates and the old sync state.	SyncManager.js:81-100, 152, 460
recoverCorruptedFiles	Boolean	Option of the resync request (HttpController.js:963-964). In a hard resync, a doc whose history content is corrupted is removed and added again from docstore (SyncManager.js:942-951).	SyncManager.js:153, 461
history	Array of Object	The last 100 states, newest first, each { syncState, timestamp }; syncState holds the five fields above.	SyncManager.js:41, 202-208
lastUpdated	Date	Time of the last write.	SyncManager.js:209
resyncCount	Number	Number of resyncs started.	SyncManager.js:215
resyncPendingSince	Date	When the running resync started; removed when it finishes.	SyncManager.js:217, 224-225
expiresAt	Date	Set to 90 days after a resync finishes, removed while one runs. The TTL index deletes the document at this time.	SyncManager.js:42, 216, 221-223
stuckClearCount	Number	How often a stuck resync was cleared and restarted; removed when a resync finishes.	SyncManager.js:261, 226
lastStuckClearAt	Date	Time of the last such clear.	SyncManager.js:262, 227
lastStuckDocPaths	Array of String	Doc paths still pending at the last clear.	SyncManager.js:262, 228

Who Writes It

Code	What it does
services/project-history/app/js/SyncManager.js:195-249	setResyncState: upsert of the state. Called when a resync starts (SyncManager.js:163) and while project-history processes the resync updates (services/project-history/app/js/UpdatesProcessor.js:637, 784).
SyncManager.js:257-266	_recordStuckClearInSyncState: counts a stuck resync that is cleared.
SyncManager.js:251-255	clearResyncState: deletes the document, at the start of a hard resync (SyncManager.js:87), on project-history DELETE /project/:project_id (services/project-history/app/js/HttpController.js:1363-1396), and when a structure-only resync has to start over (SyncManager.js:705-716).
SyncManager.js:273-289	clearResyncStateIfAllAfter: deletes a finished state whose history is all newer than a date. Nothing in app/js calls it.
SyncManager.js:171-181	cloneResyncState: copies the state to another project, for project-history POST /project/:project_id/clone (HttpController.js:57, 124), which nothing in web calls.

A resync is started by project-history POST /project/:project_id/resync (HttpController.js:950-994), by the retry cron jobs (services/project-history/app/js/RetryManager.js:156-179, server-ce/config/crontab-history), and by project-history itself when processing updates needs one (UpdatesProcessor.js:75, 224).

Who Reads It

Code	What it does
SyncManager.js:187-193	getResyncState: loads the state. Used to decide whether a resync is running before a new one starts (SyncManager.js:106-149), to skip ordinary updates while it runs (SyncManager.js:291-300), and after a flush to report a running resync (UpdatesProcessor.js:463-466).
HttpController.js:533-545	GET /project/:project_id/resync-pending: whether a resync is running or stuck.
HttpController.js:563-585	GET /project/:project_id/debug-info: the sync state with the failure record.

Indexes

Name	Keys	Options	Migration
project_id_1	{ project_id: 1 }		tools/migrations/20190912145021_create_projectHistorySyncState_indexes.mjs
expiresAt_1	{ expiresAt: 1 }	TTL, expireAfterSeconds: 0	tools/migrations/20190912145021_create_projectHistorySyncState_indexes.mjs

Notes

• A resync is running while resyncProjectStructure is true or resyncDocContents is not empty (SyncManager.js:565-567). It counts as stuck after 4 hours, and as permanently stuck after 5 clears (SyncManager.js:43-44, 569-585).
• When a resync finishes, project-history also sets overleaf.history.lastResyncedAt on the project (SyncManager.js:239-248).