|
Size: 6037
Comment:
|
Size: 9038
Comment: more anchor tweaks
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| #acl _All:read | #acl All:read |
| Line 5: | Line 5: |
| {{{#!wiki warning The docker image for Compilers 2026 is not yet published, and this page has an example from CSCI423 (my simulation course) that I need to replace. Obviously this is a work in progress, but it'll be polished off RSN. }}} |
|
| Line 13: | Line 7: |
| A student has three choices for their development environment: | A student has four choices for their development environment, we'll call these [[#docker|command line docker]], [[#compgrading|COMPGRADING]], [[#devcontainer|VsCode dev container]], and "[[#freestyle|freestyle]]". |
| Line 15: | Line 9: |
| 1. <<Anchor(dockerimage)>>"Pull" the course Docker image at https://hub.docker.com/r/khellmanatmines/comp-student, and do their development and testing on a virtual machine. | <<Anchor(docker)>><<BR>> = Command Line Docker = "Pull" the course Docker image and development in a virtual machine (you will need to have docker installed for your OS already) 1. "Pull" the course Docker image at https://hub.docker.com/r/khellmanatmines/comp-student |
| Line 19: | Line 16: |
| Then you will need to install the language toolchain of your choice, if your project will use Java and Scala: | Next, you will need to set up the compilers and tools for your preferred language, spin up the docker container with `root` access: |
| Line 22: | Line 20: |
| vmID# /usr/local/install-scripts/install-java.sh ... vmID# /usr/local/install-scripts/install-scala.sh ... |
|
| Line 27: | Line 21: |
| {{{#!wiki important You can install whatever you like on your docker vm, but the Gradescope grading machines will use '''only the `/usr/local/install-*.sh` scripts''' to setup your build environment. }}} {{{#!wiki tip Look in `/usr/local/install-scripts` for the available languages you can use to complete your project. If you don't find a language you'd like to use, '''let me know, I'll likely add it!.''' }}} Once you have your toolchains installed, `exit` the virtual machine and commit your image for future project work. {{{#!plain $ docker commit vmID compgrading sha256:6ee998 blah blah blah $ docker image list REPOSITORY TAG IMAGE ID CREATED SIZE compgrading latest 6ee998a8a914 16 seconds ago 1.8GB }}} |
and follow [[#installtoolchain|these instructions]] for package installations. |
| Line 49: | Line 31: |
| Now you can complete all your course projects as the `blaster` user on the virtual machine. {{{ $ docker run -it compgrading runuser -l blaster blaster@vmID $ echo $COMPGRADING /COMPGRADING blaster@vmID $ /COMPGRADING/Random 423 | head -n 3 >test.dat blaster@vmID:~$ cat test.dat 0.53229350689799 0.91892032278702 0.11796685447916 }}} 1. <<Anchor(linuxhost)>>Students can use their own Linux environment with the course's grading scripts and utils installed within it. |
Test your `blaster` environment by following [[#testsetup|these instructions]]. Now you can complete all your course projects as the `blaster` user on the virtual machine. <<Anchor(compgrading)>><<BR>> = COMPGRADING = Students can use their own Linux environment with the course's grading scripts and utils installed within it. |
| Line 71: | Line 48: |
| a. And make sure the binaries work {{{#!plain $ /your/path/to/somewhere/COMPGRADING/Random 423 | head -n 3 0.53229350689799 0.91892032278702 0.11796685447916 }}} All the binaries should be built with static linking to avoid shared library dependencies. If you find one that causes problems, please let the instructor know ASAP. 1. <<Anchor(nonlinux)>>Lastly, you ''could'' simply develop your code on a non-Linux box, and package it up appropriately in a ZIP for submission. But you won't be able to easily run the project's "grader script" so your test-debug-edit cycle through gradescope is going to be painful. I do '''not''' recommend this approach, but if you choose to develop this way you will most certainly want the downloads from [[Assignments/TraceFiles|the trace files page]]. |
a. Lastly, test your `blaster` environment by following [[#testsetup|these instructions]]. All the binaries should be built with static linking to avoid shared library dependencies. If you find one that causes problems, please let the instructor know ASAP. <<Anchor(devcontainer)>><<BR>> = dev container = This is particularly oriented toward [[https://github.com/microsoft/vscode|VsCode]].<<FootNote(Many thanks to MK (Compilers '26) for helping debug the `.devcontainer.json` file.)>> This approach uses the [[https://github.com/Microsoft/vscode-remote-release|Dev Containers]] Microsoft extension, which you likely have installed from some other course and if not, !VsCode will suggest you install it. {{{#!wiki tip If you are installing the extension the first time, it may require setting up WSL for docker. }}} {{{#!wiki warning Unfortunately, my windows machine is soo pathetic I cannot reliably test this setup; and the Mines Windows installs don't let users instantiate WSL. Any and all clarifications, corrections or updates from students would be very much appreciated. }}} a. Make sure the docker app (server) is running. a. Download <<DjHR(comp-dev-container.zip)>> and unravel it on your docker and !VsCode hosting machine where you want to keep all your project work. This ZIP contains a single file in a Unix "hidden" directory: `.devcontainer/devcontainer.json` a. Start !VsCode, make sure the docker container extension by `ms-azuretools` is installed and ready to roll a. Use the !VsCode `File` menu and open the ''folder containing'' the hidden directory `.devcontainer` a. !VsCode should spin up a docker instance for the container and files (directories) created within the workspace should show up magically in the docker container. To install tool chain (language) packages on your !VsCode dev-container environment: a. Start an OS terminal session and run the `docker ps --all` command to find the running container's vmID. a. In your OS terminal, connect to a `root` shell on the container with `$ docker exec -it vmID /bin/bash` a. Now, follow [[installtoolchain|these instructions]] to setup your preferred language environment. Be sure to commit your changes when done! {{{#!wiki tip The dev container is setup so that !VsCode access to the container is via the `blaster` user, but `docker exec` command line access is via `root`. Which is why you need a side-band administrative terminal for installs. The `blaster` user doesn't have access to typical `su` and `sudo` commands in the Ubuntu image from Gradescope. }}} <<Anchor(freestyle)>><<BR>> = Freestyle = Lastly, you ''could'' simply develop your code on a non-Linux box and package it up appropriately in a ZIP for submission. But you won't be able to easily run the project's "grader script" so your test-debug-edit cycle through gradescope is going to be painful. I do '''not''' recommend this approach. <<Anchor(installtoolchain)>><<BR>> = Installing tool chain (language) packages = You will need to install the language toolchain of your choice into your development docker image, if your project will use Java and Scala: {{{#!plain vmID# /usr/local/install-scripts/install-java.sh ... vmID# /usr/local/install-scripts/install-scala.sh ... }}} {{{#!wiki important You can install whatever you like on your docker vm, but the Gradescope grading machines will use '''only the `/usr/local/install-*.sh` scripts''' to setup your build environment. }}} {{{#!wiki tip Look in `/usr/local/install-scripts` for the available languages you can use to complete your project. If you don't find a language you'd like to use, '''let me know, I'll likely add it!.''' }}} Once you have your toolchains installed, `exit` the virtual machine and commit your image for future project work. {{{#!plain $ docker commit vmID compgrading sha256:6ee998 blah blah blah $ docker image list REPOSITORY TAG IMAGE ID CREATED SIZE compgrading latest 6ee998a8a914 16 seconds ago 1.8GB }}} <<Anchor(#testsetup)>><<BR>> = Test your setup = Except for the "Freestyle" setup, you can test to make sure the Compiler's environment for `grader.sh` scripts is set up correctly by trying this command in docker container `blaster` shell: {{{ blaster@vmID $ echo $COMPGRADING : blaster@vmID $ "${COMPGRADING}/emule" -s 425 -repl 3 <( echo init @0 ; echo rand f0; echo emit f0 ; echo return; ) OUTPUT 0.77897 OUTPUT 0.66379 OUTPUT 0.35938 }}} The `:` placeholder above should be either `/COMPGRADING` or the path you unrolled `COMPGRADING.tar.bz2` at for the `COMPGRADING` style of development environment. |
| Line 98: | Line 141: |
| Line 108: | Line 152: |
| I suspect `/workspace` is a VSCode-ism. | I suspect `/workspace` is a !VSCode-ism. |
Programming projects for this course are graded with https://www.gradescope.com using a standard Ubuntu base environment. Graded work must meet the course submission requirements.
A student has four choices for their development environment, we'll call these command line docker, COMPGRADING, VsCode dev container, and "freestyle".
Command Line Docker
"Pull" the course Docker image and development in a virtual machine (you will need to have docker installed for your OS already)
"Pull" the course Docker image at https://hub.docker.com/r/khellmanatmines/comp-student
docker pull khellmanatmines/comp-student
Next, you will need to set up the compilers and tools for your preferred language, spin up the docker container with root access:
docker run -it khellmanatmines/comp-student /bin/bash
and follow these instructions for package installations.
If you forget the vmID or need to jump back into an exited container, use
$ docker ps --all ... $ docker start -ai vmID
to resume the session. The ps --all command will list all your interned machines, the first column CONTAINER ID should be used in the docker start command as the vmID.
Test your blaster environment by following these instructions.
Now you can complete all your course projects as the blaster user on the virtual machine.
COMPGRADING
Students can use their own Linux environment with the course's grading scripts and utils installed within it.
Download COMPGRADING.tar.bz2 to your Linux box
- Unravel the tarball somewhere...
somewhere$ tar xjf ~/Downloads/COMPGRADING.tar.bz2
(Your path for somewhere is whereever you'd like it to be.)
Set COMPGRADING in your environment --- you should really do this in your login rc file: .bashrc most likely!
export COMPGRADING="/your/path/to/somewhere/COMPGRADING"
Lastly, test your blaster environment by following these instructions.
All the binaries should be built with static linking to avoid shared library dependencies. If you find one that causes problems, please let the instructor know ASAP.
dev container
This is particularly oriented toward VsCode.1 This approach uses the Dev Containers Microsoft extension, which you likely have installed from some other course and if not, VsCode will suggest you install it.
If you are installing the extension the first time, it may require setting up WSL for docker.
Unfortunately, my windows machine is soo pathetic I cannot reliably test this setup; and the Mines Windows installs don't let users instantiate WSL. Any and all clarifications, corrections or updates from students would be very much appreciated.
- Make sure the docker app (server) is running.
Download comp-dev-container.zip and unravel it on your docker and VsCode hosting machine where you want to keep all your project work. This ZIP contains a single file in a Unix "hidden" directory: .devcontainer/devcontainer.json
Start VsCode, make sure the docker container extension by ms-azuretools is installed and ready to roll
Use the VsCode File menu and open the folder containing the hidden directory .devcontainer
VsCode should spin up a docker instance for the container and files (directories) created within the workspace should show up magically in the docker container.
To install tool chain (language) packages on your VsCode dev-container environment:
Start an OS terminal session and run the docker ps --all command to find the running container's vmID.
In your OS terminal, connect to a root shell on the container with $ docker exec -it vmID /bin/bash
Now, follow these instructions to setup your preferred language environment. Be sure to commit your changes when done!
The dev container is setup so that VsCode access to the container is via the blaster user, but docker exec command line access is via root. Which is why you need a side-band administrative terminal for installs. The blaster user doesn't have access to typical su and sudo commands in the Ubuntu image from Gradescope.
Freestyle
Lastly, you could simply develop your code on a non-Linux box and package it up appropriately in a ZIP for submission. But you won't be able to easily run the project's "grader script" so your test-debug-edit cycle through gradescope is going to be painful. I do not recommend this approach.
Installing tool chain (language) packages
You will need to install the language toolchain of your choice into your development docker image, if your project will use Java and Scala:
vmID# /usr/local/install-scripts/install-java.sh ... vmID# /usr/local/install-scripts/install-scala.sh ...
You can install whatever you like on your docker vm, but the Gradescope grading machines will use only the /usr/local/install-*.sh scripts to setup your build environment.
Look in /usr/local/install-scripts for the available languages you can use to complete your project. If you don't find a language you'd like to use, let me know, I'll likely add it!.
Once you have your toolchains installed, exit the virtual machine and commit your image for future project work.
$ docker commit vmID compgrading sha256:6ee998 blah blah blah $ docker image list REPOSITORY TAG IMAGE ID CREATED SIZE compgrading latest 6ee998a8a914 16 seconds ago 1.8GB
Test your setup
Except for the "Freestyle" setup, you can test to make sure the Compiler's environment for grader.sh scripts is set up correctly by trying this command in docker container blaster shell:
blaster@vmID $ echo $COMPGRADING : blaster@vmID $ "${COMPGRADING}/emule" -s 425 -repl 3 <( echo init @0 ; echo rand f0; echo emit f0 ; echo return; ) OUTPUT 0.77897 OUTPUT 0.66379 OUTPUT 0.35938
The : placeholder above should be either /COMPGRADING or the path you unrolled COMPGRADING.tar.bz2 at for the COMPGRADING style of development environment.
Hints for developing in a docker image
Viewing PDFs
The grader.sh scripts for some projects produce PDFs of parse trees. It is important to inspect these to make sure your submission is working as it should. However, if you don't have a setup where you can simply "file browse" to the directory in the docker image and view the PDFs in your preferred GUI, this can be a bit klunky to do over and over and over during development and testing.
I'm sure there are only a bazillion ways to do this on your favorite OS. Google knows of course... Here is my "terminal-centric" solution.
After running grader.sh, in another terminal on your machine use
$ docker exec vmID runuser -u blaster -- /bin/sh -c '( cd && cd path/to/my/project && tar cf - _*.pdf)' | tar xf -
which will dump all the grader.sh script PDFs in your current directory (you could of course use zip instead, choose your weapon of choice). If you use an auto-refreshing PDF viewer (like gv(1)) you can now open the PDF plot of interest and it will reload whenever the PDF transfer occurs. Good stuff.
If you don't like having to "up-arrow" and re-issuing the docker exec command all the time, consider:
$ while sleep 4 ; do docker exec ...<your transfer command> ; done
Using the Docker container with VSCode on MacOS
(Many thanks to NatalieR, CSCI423 Fall 25 for these insights).
- Run one time
docker run --platform linux/amd64 -it --name compdev \ -v /Users/USERNAME/Desktop/CSCI425/Project1:/workspace khellmanatmines/comp-student \ runuser -l blaster -c "cd /workspace && bash -l"
Where /Users/USERNAME/Desktop/Project1:/workspace is specific to your own Mac setup, compdev is the name of the container being created, and /workspace on the container should probably pre-exist and be owned by the blaster user.
I suspect /workspace is a !VSCode-ism.
- You can reconnect to the container with
docker start -ai compdev
Many thanks to MK (Compilers '26) for helping debug the .devcontainer.json file. (1)