v2.4.0.0 Adding or Modifying Pools LSF

It’s not uncommon to refine or otherwise modify configurations over time. Soon there will be a CLI tool to obviate the need for the following manual steps, which are the current steps required for reconfiguration after the initial integration has been pushed into production.

1. Navigate to the exostellar directory where the configuration assets reside:

   1. cd ${LSF_RC_CONF_DIR}/exostellar

2. Setup a timestamp folder in case there’s a need to rollback:

   1. PREVIOUS_DIR=$( date +%Y-%m-%d_%H-%M-%S ) mkdir ${PREVIOUS_DIR}

3. Place the contents of the exostellar directory in the timestamp directory:

   1. mv * ${PREVIOUS_DIR}

4. Make a new json folder and copy the env.json and profile.json:

   1. mkdir json cp ${PREVIOUS_DIR}/json/env0.json ${PREVIOUS_DIR}/json/profile0.json json/ cd json mv env0.json env1.json mv profile0.json profile1.json

5. Edit env1.json as needed, e.g.:

   1. Add more pools if you need more CPU-core or Memory options availble in the partition.

   2. Increase the node count in pools.

   3. Environment Configuration Information for reference.

6. Likely, profile0.json may not need any modification.

   1. Profile Configuration Information for reference.

7. Validate the JSON asset with jq:

   1. jq . env1.json

8. You will see well-formatted JSON if jq can read the file, indicating no errors. If you see an error message, that means the JSON is not valid.

9. When the JSON is valid, the file can be pushed to the MGMT_SERVER:

   1. curl -d "@env1.json" -H 'Content-Type: application/json' -X PUT http://${MGMT_SERVER_IP}:5000/v1/env

10. If the profile was changed, validate it with the quick jq test.

    1. jq . profile1.json

11. Push the changes live:

12. curl -d "@profile1.json" -H 'Content-Type: application/json' -X PUT http://${MGMT_SERVER_IP}:5000/v1/profile

13. Grab the assets from the MGMT_SERVER:

14. curl -X GET http://${MGMT_SERVER_IP}:5000/v1/xcompute/download/lsf -o lsf.tgz

    1. If the EnvName was changed (above in Edit the LSF Environment JSON for Your Purposes - Step 2 ), then the following command can be used with your CustomEnvironmentName :

    2. curl -X GET http://${MGMT_SERVER_IP}:5000/v1/xcompute/download/lsf?envName=CustomEnvironmentName -o lsf.tgz

15. Unpack them into the exostellar folder:

16. tar xf lsf.tgz -C ../ cd .. mv assets/* . rm assets chown lsfadmin.root -R ../exostellar

17. Integration steps are complete and a job submission to the new partition is the last validation:

    1. As a user, navigate to a valid job submission directory and launch a job as normal, but be sure to specifiy the new partition:

       1. bsub -q xio < job-script.sh