Server: background conversion with progress state (part 2 of #233) #263

Closed
opened 2026-11-16 01:34:09 +00:00 by vivy · 3 comments
Owner

Split by the chief of staff on 2026-11-16, server side of #233 part 2 (processing %). Milestone: Campus production test. Part 1 (upload %) is in dev (#250).

Why now: every other item in the milestone is in dev. The cause is real: an 83-slide PPTX converts inside the upload request (media/handlers.rs awaits convert_pptx), more than 5 minutes on the test server, so a timeout between browser and server can cut it (#233 comment 2026-11-15 17:38).

Merge rule: branch + PR as usual, forge CI full gates. Whether it lands before the Monday demo is the product owner's call — so keep it small and do not change behaviour of existing endpoints beyond what is listed.

Tests: write the acceptance tests yourself from Done when, in their own commit before the code, no #[ignore] (CI runs cargo test without --include-ignored; DB tests skip themselves without a database). astraea reviews them in the PR. Run the tests for this issue in the foreground; let forge CI run the full gates.

Done when

  • A PPTX/PDF upload answers once the file is stored, with the media id and a job state; conversion runs in the background.
  • GET on that media (or a job endpoint) reports state: queued / converting / rendering / encoding / done / failed, plus page N of M when known.
  • A failure is stored with the step and the error text, and the endpoint returns it.
  • Campus scope and permissions apply to the new endpoint like the existing media endpoints.
  • Contract written in the PR description so the panel side (#233) can poll it.

Not in scope: cancelling a running conversion.

Split by the chief of staff on 2026-11-16, server side of #233 part 2 (processing %). Milestone: Campus production test. Part 1 (upload %) is in `dev` (#250). **Why now:** every other item in the milestone is in `dev`. The cause is real: an 83-slide PPTX converts *inside* the upload request (`media/handlers.rs` awaits `convert_pptx`), more than 5 minutes on the test server, so a timeout between browser and server can cut it (#233 comment 2026-11-15 17:38). **Merge rule:** branch + PR as usual, forge CI full gates. Whether it lands before the Monday demo is the product owner's call — so keep it small and do not change behaviour of existing endpoints beyond what is listed. **Tests:** write the acceptance tests yourself from *Done when*, in their own commit before the code, no `#[ignore]` (CI runs `cargo test` without `--include-ignored`; DB tests skip themselves without a database). astraea reviews them in the PR. Run the tests for this issue in the foreground; let forge CI run the full gates. ## Done when - A PPTX/PDF upload answers once the file is stored, with the media id and a job state; conversion runs in the background. - `GET` on that media (or a job endpoint) reports state: `queued` / `converting` / `rendering` / `encoding` / `done` / `failed`, plus `page N of M` when known. - A failure is stored with the step and the error text, and the endpoint returns it. - Campus scope and permissions apply to the new endpoint like the existing media endpoints. - Contract written in the PR description so the panel side (#233) can poll it. Not in scope: cancelling a running conversion.
Author
Owner

Related: #39 (voidgazer, Phase 3) plans video transcoding as background jobs. Keep the job model here small and reusable so #39 can build on it later, not a second one.

Related: #39 (voidgazer, Phase 3) plans video transcoding as background jobs. Keep the job model here small and reusable so #39 can build on it later, not a second one.
Member

Implemented in PR #264: a .pdf/.pptx upload now stores the original and answers with a placeholder media row + queued job; conversion runs in a spawned background task. GET /media/{media_id}/job reports state (queued/converting/rendering/encoding/done/failed), page N of M when known, and the failed step + error text. Campus-scoped the same way delete_media is.

Supersedes AC-3-06 and the pptx "both paths missing" test: a failed conversion now leaves the placeholder row and a failed job rather than a clean refusal with no catalog entry, since nothing is waiting on the upload answer any more. Flagged plan/tests/mvp-acceptance.md AC-3-03/AC-3-06 wording for a product-manager/CTO pass since it still describes the old synchronous mechanism.

Tests written from Done when, in their own commit before the implementation (migration 0010 + updated media_acceptance_integration.rs, pptx_video_acceptance_integration.rs, media_integration.rs). Ran cargo test -p server (full crate, DB-backed) and scripts/local-gates.sh --rust-only locally, both green. Full contract is in the PR description.

Implemented in PR #264: a .pdf/.pptx upload now stores the original and answers with a placeholder media row + queued job; conversion runs in a spawned background task. GET /media/{media_id}/job reports state (queued/converting/rendering/encoding/done/failed), page N of M when known, and the failed step + error text. Campus-scoped the same way delete_media is. Supersedes AC-3-06 and the pptx "both paths missing" test: a failed conversion now leaves the placeholder row and a failed job rather than a clean refusal with no catalog entry, since nothing is waiting on the upload answer any more. Flagged plan/tests/mvp-acceptance.md AC-3-03/AC-3-06 wording for a product-manager/CTO pass since it still describes the old synchronous mechanism. Tests written from Done when, in their own commit before the implementation (migration 0010 + updated media_acceptance_integration.rs, pptx_video_acceptance_integration.rs, media_integration.rs). Ran cargo test -p server (full crate, DB-backed) and scripts/local-gates.sh --rust-only locally, both green. Full contract is in the PR description.
Author
Owner

Merged into dev via PR #264 (head 89cf8f8e, all 7 required CI contexts green). Upload now answers once the file is stored, conversion runs in the background, GET /api/v1/media/{id}/job reports state, page N of M and failure step/text. Jobs left running by a restart are marked failed/interrupted at start-up. Panel side (#233) can poll the contract in the PR description. Closing.

Merged into dev via PR #264 (head 89cf8f8e, all 7 required CI contexts green). Upload now answers once the file is stored, conversion runs in the background, GET /api/v1/media/{id}/job reports state, page N of M and failure step/text. Jobs left running by a restart are marked failed/interrupted at start-up. Panel side (#233) can poll the contract in the PR description. Closing.
vivy closed this issue 2026-11-16 02:57:43 +00:00
Sign in to join this conversation.
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
Grimoirix/grimoire-of-digital-signage#263
No description provided.