PRINTDECKDOCS

Architecture · Chapter 6

Builds from GitHub

A project can link one GitHub repository. At each push, the Action sends the parts of the commit. The owner chooses the plates, and the Action slices them.

6.1 The flow

  1. Push. A person pushes a branch or a tag. The workflow .github/workflows/printdeck.yml starts in the repository that gets the push: the repository of the project, or the fork of a member.
  2. Plan run. The job plan reads printdeck.yml and runs its generate step. It finds the parts that changed. Then it sends the plan, with an STL model of each file of each part and the PDFs of its docs.
  3. Plan page. The owner opens /projects/<slug>/plans/<id>, and chooses FDM or resin first. Each part has one row, with a chip for each of its variants, for example a fine and a standard quality. The parts that changed are checked at first, and a release plan checks all the parts. The owner can change the checks, the variants and the copies of each part.
  4. Plates. The page suggests the plates, and checks that each plate fits on its bed. The owner can move a part to another plate or to a new plate.
  5. Slice. The owner taps Slice. The API makes a build with the state "slicing". The GitHub App of the service starts the workflow on the default branch, with the input build.
  6. Slice run. The job slice gets the plates from the API, and checks out the commit of the plan. It slices each plate, sends the facts of the files, uploads the files, and marks the build done.
  7. Build page. /projects/<slug>/builds/<id> shows each plate with its facts: the profile with its version, the material, the time, the grams or ml, the layers, the file and the slicer. A 3D preview draws the moves of the G-code of a plate, layer by layer, with the colors of Cura. While the build slices, the page shows what the slice run does, and reads the build again each 10 seconds.
  8. Approval. The owner approves each plate, or rejects it with a reason. The API records the state, the name and the time. The owner then sends an approved plate to a printer: it goes to the queue of the deck (section 7.3).
ActionGitHub ActionsAPILambdaBucketS3Owner pagebrowserplan run: a push, a pull request or a tag1POST /api/v1/plansOIDC token, the parts of printdeck.ymlcheck the token, record its jti;find the project of the repository2put x/<p>/plans/<id>.json3plan ID, grant for x/<p>/plans/<id>/4POST the models (STL) and PDFs, with the grant5POST plans/<id>/done, a new tokencheck the models; mark the plan ready;changed waiting plates: replacedplan page: the owner chooses the parts and the plates6GET projects/<p>/plans/<id>, session7parts, sizes, beds, model paths, cookies8GET /x/<p>/plans/<id>/*.stl, cookie3D view of a part;plates, fit check, bed view9POST projects/<p>/plans/<id>/buildsthe plates (owner only)check the plate rules;put builds/<id>.json: slicingThe GitHub App starts the slice run. With no App,the build page shows a gh workflow run command.
Figure 21. The plan run and the plan page. Each request of the Action has a new OIDC token. The orange steps upload with a grant, or make a build.
ActionGitHub ActionsAPILambdaBucketS3Owner pagebrowserslice run: workflow_dispatch with the input build1GET /api/v1/builds/<id>token of printdeck.yml on maincheck the token, jti andjob_workflow_ref; same repository2the commit and the plates3POST builds/<id>/started: the run IDcheck out the commit;CuraEngine or PrusaSlicer4POST builds/<id>/filesname, size, SHA-256, time, grams or ml, layers5grant for x/<p>/builds/<id>/6POST the G-code of each plate7POST builds/<id>/donecheck the files;build: ready, plates: waitingbuild page: the approval8GET projects/<p>/builds/<id>9plates, facts, profile with its version10POST builds/<id>/plates/<n>/approveor reject with a reason (owner)plate: approved or rejected,with the name and the time
Figure 22. The slice run and the approval. Each request on /api/v1/builds/ needs a token of the workflow printdeck.yml on main or master.

6.2 The plans and the builds

The bucket keeps each plan and each build as a JSON record of the project, with a random ID of 10 bytes. The overview of the project page shows the open pull requests and the newest plan of main (section 5.5). The overview hides a closed or merged pull request. The API asks GitHub with the App for the state of each pull request, at most each 10 minutes for an open one and each 6 hours for a closed one. The Builds tab and the page /projects/<slug>/plans show only the plans of pull requests and of tags, for example v1.2 or release/1.2.

RecordKeyWhat it has
Planx/<project>/plans/<id>.jsonThe repository, the commit, the ref, the kind, the pull request, the tag or the branch, the run ID, and "ready". A plan of a fork also has the fork and the GitHub username of its member. For each part: the name, the variant, the files, the count, the printer, the process, the profile with its version, the material, prebuilt, alone, changed, the digest, the model files, the size of the model, the bed and the margin. The plan also has its PDFs (name, file, size, pages, SHA-256), and for a pull request the base plan and the changes of its models.
Plan modelsx/<project>/plans/<id>/part<i>-<j>.stlOne STL for each file of each part, at the middle, for the 3D views. A 3MF file loses its support blockers here. Each PDF is doc<n>.pdf, with a PNG of its first page, doc<n>.png (600 pixels wide).
Buildx/<project>/builds/<id>.jsonThe plan, the repository, the commit, the state, the error, the run ID and the plates. Each plate has the printer, the process, the profile, the material and the parts with their counts. After the slice, it has the file, the size, the SHA-256, the time, the grams or ml, the layers, the layer height and the slicer. The approval adds the state, the name, the time and the reason. A plate that the owner sends has the printer and the times of the send and of the take. A plate with a picture for the comment has its public address.
Build filesx/<project>/builds/<id>/plate-<n>.gcodeThe sliced file of each plate. A resin plate has a .cbddlp file, for example plate-1.cbddlp: the slice of PrusaSlicer, or the prebuilt file.

The kind of a plan comes from the ref of the token. The body of the request cannot change it.

  • test: a branch with an open pull request. The API asks GitHub with the App: a branch of the repository with an open pull request, with the commit of the token as its head, makes the plan of that pull request. A plan of a member's fork is always of this kind (section 6.8).
  • main: the branch main or master.
  • branch: another branch, with no open pull request.
  • release: a tag, for example v1.2.

The commit comes from the token. A plan of a fork must use the commit of its token. It must have 40 hex digits. A plan has 1 to 50 parts. A part has at most 50 files and 1 to 100 copies. A build has 1 to 20 plates.

BUILD (builds/<id>.json, state)slicingmade by Slicereadyall files are upfailedwith the errordonefailedA new build of the same plan has its own state.PLATE (each plate of a build)waitingfor the ownerapprovedname, timerejectedname, time, reasonreplaceda newer plan changed itApproveRejecta new plan that changesa part of the plate
Figure 23. The states of a build and of its plates. A new run of a failed build sets it back to "slicing". Slice again on the build page makes a new build with the same plates.

When a new plan of a pull request or a branch is done, the API marks the waiting plates of the older plans of the same pull request or branch as "replaced", when a part of the plate is gone or changed. The Action sends a digest of each part: the SHA-256 of its entry, its files and its profile. With no digest, the mark changed, the profile and the material decide. The approve and reject routes refuse a replaced plate with 409. They also refuse a plate of a build that is not ready.

6.3 The plates

  • The parts of one plate have one printer, one profile and one material. The API compares each part with the first part of the plate, and refuses a mix with 400.
  • A part with alone: true or a prebuilt file prints alone on its plate. The API refuses it on a plate with other parts. A prebuilt part has 1 copy.
  • The suggestion puts the checked parts with the same printer, profile and material on one plate while they fit. A part that does not fit gets a new plate.
  • The fit check uses the grid of curaslice.py: the objects in rows, each with the room for its brim or skirt, and 10 mm from the edges of the bed. The plan has the bed and the room of each part. A plate that does not fit shows how many objects fit, and Slice stays off.
  • Each plate has a view of the bed from above, with the objects at the places of the grid. The 3D view shows the models of one part on the bed of its printer.
  • One object alone goes to the middle of the bed. The plan page does not compare the settings: of the parts. A plate with two values of a plate setting fails in the slice run.

6.4 The slicer

The slice run slices each FDM plate with curaslice.py and CuraEngine 5.13.0. curaengine.sh downloads the official AppImage of Cura 5.13.0 from GitHub and checks its SHA-256. Then it extracts CuraEngine and its libraries. The cache of GitHub Actions keeps them, so only the first slice run downloads the AppImage (330 MB).

The slice run slices each resin plate with resinslice.py and PrusaSlicer 2.8.1. prusaslicer.sh downloads the official AppImage of PrusaSlicer (99 MB) and checks its SHA-256, and the cache keeps it too. PrusaSlicer slices the parts of the plate as one model, with its supports and its pad. resinslice.py writes the layers as a .cbddlp file of version 4 for the ChiTu board of the Phenom, as ChiTuBox 1.4.0 writes it.

The Cura app works out the value of each setting before it calls CuraEngine. curaslice.py does the same work with the rules of Cura 5, in the Python standard library:

  • The stack of the printer, top first: the settings of the part, the quality changes (the profile), the quality, the material (extruder only), the definition changes, and the definition with its parents.
  • The extruder stack falls through to the global stack. A setting that is not settable per extruder comes from the global stack.
  • A formula gets the values of its names from the stack that asks. The functions are those of Cura, for example extruderValue and resolveOrValue.
  • The {key} tokens of the start and end G-code come from the first extruder.
  • The script writes the real header (time, filament, size) from the log of CuraEngine, as the Cura app does.

The folder profiles/taz-workhorse-se of Printdeck is the complete Cura stack of the TAZ Workhorse with the SE tool head. slicer/export_cura_profile.py copies it from the Cura app of the owner, and stops when a file has a secret. A repository copies the folder of each profile that it uses into its own profiles/. So a build depends only on its commit, and a pull request can change a profile. The folder profiles/peopoly-phenom has resin.json: the printer (276.48 x 155.52 mm, 3840 x 2160 pixels, a margin of 4 mm) and the resin profile abs-like-0.05, with the settings of ChiTuBox for the resin.

A setting of a part that Cura allows for each mesh stays with its part. Another setting of a part applies to the whole plate. A 3MF file keeps the per-mesh settings of Cura, for example a support blocker. A part with a prebuilt file, for example a .cbddlp file from ChiTuBox, needs no slice. The Action copies that file as the file of the plate.

The check against Cura. profiles/README.md compares a slice of the Cura app with a slice of curaslice.py of the same STL (PLA 0.10 Fine, roll 4, a brim of 12 lines, tree supports). The print time (12399 s), the filament (2.07922 m), the size and the layers (738) are the same. Of the 211,427 lines of moves, 245 places differ by 0.001 to 0.005 mm.

6.5 The profile version in each G-code

Each profile has a version: the first 12 hex digits of the SHA-256 of the paths and the contents of all the files of its folder. A change to any file of the folder gives a new version. The plan has the version for each part, for example taz-workhorse-se/pla-0.10-fine@44606b4e58b6.

The header of each G-code has the line ;Printdeck profile: with this value. It also has ;Printdeck title:, ;Printdeck build: and ;Printdeck commit:, and the material lines of the Printdeck plugin for Cura. The API keeps the profile from the slice run only when it starts with the name of the profile of the plan, before @. The build page shows the profile with its version for each plate.

6.6 The Action: printdeck-action

The Action is the public repository beetlebugorg/printdeck-action, under AGPL-3.0. A workflow uses it as beetlebugorg/printdeck-action@v1. It is a composite action with six files: action.yml, printdeck_action.py, curaslice.py, curaengine.sh, resinslice.py and prusaslicer.sh. Its inputs are mode (plan or slice), build, config (default printdeck.yml) and api (default https://printdeck.beetlebug.org).

The workflow in its README has the permissions contents: read and id-token: write, and two jobs:

  • plan runs on a push of each branch and of each tag v*. The workflow has no trigger pull_request. The run checks out the full history with no stored token, and compares a branch with its merge base with the default branch, and main with the commit before the push. A part changed when its files, its prebuilt file, the folder of its profile, or its entry in printdeck.yml changed. A tag has no base, so its plan marks no part.
  • slice runs on workflow_dispatch with the inputs build and title. The title names the plates, and GitHub shows it as the name of the run. It gets the plates, and sends its run ID to builds/<id>/started. The build page then links to the run. A failure sends builds/<id>/failed with the error.

The Action needs no secret. For each request, it asks GitHub for a new OIDC token, with the audience printdeck.beetlebug.org: the host name of the input api.

The GitHub App. The service starts each slice run with its GitHub App, when the stack has the secret <stack>-github-app (pwf github-app). For each build, the API signs a JWT of the App and finds the installation of the App on the repository. It gets an installation token for that one repository, with the permissions Actions (write) and Metadata (read). Then it reads the default branch, and starts printdeck.yml on it with the inputs build and title. A workflow from before the input title gets the build only. When GitHub refuses, the build fails with the reason. With no App, the build page shows the owner the command gh workflow run printdeck.yml -R <repo> -f build=<id>, with Copy.

6.7 Security of the builds

  1. The token. Each request of the Action has an OIDC token of GitHub in the header Authorization. The API checks its RS256 signature with the keys of GitHub, from https://token.actions.githubusercontent.com/.well-known/jwks. It keeps the keys for 1 hour, and reads them again for a key ID that it does not have. It checks the issuer, the audience (the host name of the service) and the expiry, with 1 minute of leeway. The token must have the claim repository.
  2. One use. The API writes oidc/<SHA-256 of the jti> with a conditional write that fails when the file exists. A second use of a token gets 401, and so does a token with no jti. A lifecycle rule deletes these files after 1 day.
  3. The link to the repository. The owner links a repository on the project page (PUT projects/<id>/repo). The API writes repos/<owner>/<name>.json in lower case, with the project. The repository of the token selects the project, and the project must still name that repository. Otherwise the request gets 403. A second project cannot link the same repository (409). A plan or a build of another repository gets 403.
  4. The reviewed workflow. Each request on /api/v1/builds/ needs a token with the job_workflow_ref <repo>/.github/workflows/printdeck.yml@refs/heads/main, or the same on master. A change to that file needs a push or a merge to that branch. A copy of the workflow on another branch or in a pull request can send a plan, and gets 403 on a build.
  5. Forks. The run of a fork sends a plan with the token of the fork. Section 6.8 gives the checks. A fork never fills a build: its token has another job_workflow_ref.
  6. The uploads. The grant of a plan or a build has one folder, 200 MB for each file, and 1 hour. Done checks that each named file is in the folder. Each file of a build needs a size of at most 200 MB and a SHA-256 in hex.
  7. The people. Only the owner links a repository, approves or rejects a plate, and sends a plate to a printer. The owner slices a plan, and can let a member slice too. Members see the plans and the builds. The routes that show a plan or a build refuse a deck.
  8. The App. Each installation token of the App works for one repository, with only the permission of its call: start a workflow, read the pull requests, or write the comment on a pull request. The private key of the App is in Secrets Manager. The role of the function can read that secret only when the stack has the App.

Section 4.2 lists the limits of the builds.

6.8 Pull requests from forks

A member can work in a fork of the repository of the project. The repository of the project runs nothing for a fork. The workflow of the fork runs in the fork, and asks GitHub for an OIDC token. That token names the fork, its owner, the branch and the commit. Printdeck checks the signature. Then it decides by its own rules if it trusts the token.

The fork · sam/bittlex a push of the branch door-hinge run generate, read the parts ask GitHub for an OIDC token POST /api/v1/plans token: sam/bittlex, owner sam, the commit GitHub signs the token (JWKS) GitHub API the open pull requests of bittlex (the App) Printdeck 1 The signature key, audience, time, one use 2 A member the owner of the fork joined, with that GitHub username (no GitHub call) 3 A pull request open, from that branch of the fork, with the commit of the token as head All true: the plan of PR #9, kind test, with the fork and @sam Else: 403, and no plan A fork never reaches /api/v1/builds/ bittlex · printdeck.yml on main Runs nothing for the fork. When the owner taps Slice, it slices the commit of the pull request as data. The owner approves each plate after the preview.
Figure 24. The plan of a member's fork. The fork sends the plan with its own token. Printdeck checks the token, the member and the pull request.
CheckWhat it provesWhere the facts come from
1. The signature, the audience, the time, one useGitHub made the token for Printdeck a few minutes ago, and nobody used it before.The token, and oidc/
2. The owner of the fork is a memberThe owner invited this person, the person joined, and the owner set this GitHub username.repository_owner of the token, and the members of the projects. A stranger stops here.
3. An open pull request with that commitThe fork is a fork of the repository of the project, the member opened a pull request, and the plan is of its newest commit.The GitHub API, with an installation token of the App on the repository of the project

The API looks for a member's fork only after a token of an unknown repository. planDone then accepts a token of the fork for the plan of the fork only. A push to a fork with no pull request, a pull request of a stranger, or an old commit of the branch makes no plan.

The member forks the repository, turns on the Actions of the fork once, pushes a branch and opens a pull request. Each new push of the branch sends a new plan. The owner sets the GitHub usernames on the card People of the overview (Edit), or in Settings.

6.9 What changed in the models

The page compares each changed model of a plan with the same model in the newest older plan of main or of a tag, else the newest older plan. The Action saves each model in the middle of the bed at Z 0, so the page puts the old model at the middle, or at the low or the high side of each axis, and keeps the place with the least volume that differs (a grid of 48 cells on the longest side). Then it compares the two models in a grid of 140 cells, from the crossings of the surface along Z in each column.

  • The plans list has a line for each changed model: the change of the size, and of the volume in cm³ and in %. The models with the same shape (a change of the profile or of the settings only) share one short line.
  • The plan page has a card What changed. See in 3D shows the new model faint, the added material in green and the removed material in red.

6.10 The comment on a pull request

The service keeps one comment on each pull request, with its GitHub App. The record x/<project>/prs/<n>.json has the ID of the comment, the state of the pull request, and the public copies of the photos. When the comment is gone, the service makes a new one.

  • The comment names the commit of the plan, the count of the changed parts, and the plan that it compares with. A table gives what changed in each model, with the comparison of section 6.9 in Go (cloud/modeldiff).
  • It has a picture of each changed model, at most 8: gray for no change, green for new material, red for material that is gone.
  • It shows the newest 3 builds of all the plans of the pull request. Each plate has its parts, its time, its material and its state. An FDM plate has a picture of its G-code. A print of a plate gives its result and its first photo.
  • The pictures are public files gh/<32 hex digits>.png (and .jpg for a photo), so GitHub shows them for a private repository too. The names are random.
  • The service writes the comment again when a plan is ready, when a build starts, fails or is ready, at an approval, a rejection or a send, when a deck takes a plate, and when a print of a plate ends or gets a photo.

The API writes the comment after the request, in a job. Server.Later invokes the function again with the event {"pwf":"job"}, and RunJob writes the comment in at most 50 seconds. The job has no retries, and an event waits at most 600 seconds. With no Later, or when the invoke fails, the job runs in the request, for at most 20 seconds. A failure goes to the log only.

6.11 What is not built yet

  • Commit status: the service sets no commit status on a pull request.
  • Print records: the print page and the print tiles do not show the build or the pull request of a plate.
  • Resin pictures: a resin plate has no picture in the comment on a pull request.

Back: 5 The data of a printContentsNext: 7 The print queue