Skip to content

Adjust/lustre quota from cronjob to grid daemon #137

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Martin-Rehr wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Adjust/lustre quota from cronjob to grid daemon #137

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
dc7689a
Changed pylustrequota to a generic python lustreclient extension
Martin-Rehr Nov 25, 2025
d47502f
Changed the lustre quota container from a 'execution and exit' contai…
Martin-Rehr Nov 25, 2025
1949a3b
lustreclient: Remove absolete git branch requirement
Martin-Rehr Nov 25, 2025
8c5813c
Added 'quota_update_interval' to 'migrid' generateconfs
Martin-Rehr Nov 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 14 additions & 12 deletions Dockerfile.rocky9
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,7 @@ ARG SFTP_MAX_SESSIONS=32
ARG WSGI_PROCS=25
ARG APACHE_WORKER_PROCS=256
ARG QUOTA_BACKEND=""
ARG QUOTA_UPDATE_INTERVAL=3600
ARG QUOTA_USER_LIMIT=1099511627776
ARG QUOTA_VGRID_LIMIT=1099511627776
ARG QUOTA_LUSTRE_VERSION="2.15.4"
Expand Down Expand Up @@ -1104,9 +1105,10 @@ ARG SFTP_MAX_SESSIONS
ARG WSGI_PROCS
ARG APACHE_WORKER_PROCS
ARG QUOTA_BACKEND
ARG QUOTA_LUSTRE_VERSION
ARG QUOTA_UPDATE_INTERVAL
ARG QUOTA_USER_LIMIT
ARG QUOTA_VGRID_LIMIT
ARG QUOTA_LUSTRE_VERSION

# TODO: do we still need the ~/.local/ wrapper now that update-alternatives run?
ENV PYTHONPATH=${MIG_ROOT}
Expand Down Expand Up @@ -1238,6 +1240,7 @@ RUN ./generateconfs.py --source=. \
--enable_gdp=${ENABLE_GDP} --gdp_email_notify=${GDP_EMAIL_NOTIFY} \
--gdp_id_scramble=${GDP_ID_SCRAMBLE} --gdp_path_scramble=${GDP_PATH_SCRAMBLE} \
--enable_quota=${ENABLE_QUOTA} --quota_backend="${QUOTA_BACKEND}" \
--quota_update_interval=${QUOTA_UPDATE_INTERVAL} \
--quota_user_limit=${QUOTA_USER_LIMIT} --quota_vgrid_limit=${QUOTA_VGRID_LIMIT} \
--storage_protocols="${STORAGE_PROTOCOLS}" \
--wwwserve_max_bytes=${WWWSERVE_MAX_BYTES} \
Expand Down Expand Up @@ -1564,15 +1567,14 @@ RUN if [ "${ENABLE_GDP}" = "True" ]; then \
fi; \
fi;

# Install pylustrequota
# Install lustreclient
# NOTE: Requires py3 and git next branch
RUN if [ "${ENABLE_QUOTA}" = "True" ] \
&& [ "${PREFER_PYTHON3}" = "True" ] \
&& [ "${WITH_GIT}" = "True" ] \
&& [ "${MIG_GIT_BRANCH}" = "next" ]; then \
&& [ "${WITH_GIT}" = "True" ]; then \
if [ "${QUOTA_BACKEND}" = "lustre" ] \
|| [ "${QUOTA_BACKEND}" = "lustre-gocryptfs" ]; then \
echo "install pylustrequota" \
echo "install python lustreclient extensions" \
&& dnf update -y \
&& dnf --enablerepo=crb install -y \
python3-devel.x86_64 \
Expand All @@ -1588,22 +1590,22 @@ RUN if [ "${ENABLE_QUOTA}" = "True" ] \
krb5-devel.x86_64 \
&& dnf clean all \
&& rm -fr /var/cache/dnf \
&& cd ${MIG_ROOT}/mig/src/pylustrequota \
&& cd ${MIG_ROOT}/mig/src/lustreclient \
&& git clone https://github.com/lustre/lustre-release \
&& cd ${MIG_ROOT}/mig/src/pylustrequota/lustre-release \
&& cd ${MIG_ROOT}/mig/src/lustreclient/lustre-release \
&& git checkout ${QUOTA_LUSTRE_VERSION} \
&& sh ./autogen.sh \
&& ./configure --disable-server --enable-quota --enable-utils --enable-gss \
&& make undef.h \
&& cd ${MIG_ROOT}/mig/src/pylustrequota/lustre-release/libcfs/libcfs \
&& cd ${MIG_ROOT}/mig/src/lustreclient/lustre-release/libcfs/libcfs \
&& make libcfs.la \
&& cd ${MIG_ROOT}/mig/src/pylustrequota/lustre-release/lnet/utils/lnetconfig \
&& cd ${MIG_ROOT}/mig/src/lustreclient/lustre-release/lnet/utils/lnetconfig \
&& make liblnetconfig.la \
&& cd ${MIG_ROOT}/mig/src/pylustrequota/lustre-release/lustre/utils \
&& cd ${MIG_ROOT}/mig/src/lustreclient/lustre-release/lustre/utils \
&& make liblustreapi.la \
&& cd ${MIG_ROOT}/mig/src/pylustrequota \
&& cd ${MIG_ROOT}/mig/src/lustreclient \
&& python3 -m pip install --use-pep517 . \
&& rm -rf ${MIG_ROOT}/mig/src/pylustrequota/lustre-release; \
&& rm -rf ${MIG_ROOT}/mig/src/lustreclient/lustre-release; \
fi; \
fi;
#------------------------- next stage -----------------------------#
Expand Down
4 changes: 2 additions & 2 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ initdirs: initcomposevars
mkdir -p ${LOG_ROOT}/syslog/migrid-sftp
mkdir -p ${LOG_ROOT}/syslog/migrid-webdavs
mkdir -p ${LOG_ROOT}/syslog/migrid-ftps
mkdir -p ${LOG_ROOT}/syslog/migrid-lustre-quota
mkdir -p ${LOG_ROOT}/syslog/migrid-quota

initcomposevars:
@echo "creating env variable map in docker-compose_shared.yml"
Expand Down Expand Up @@ -176,7 +176,7 @@ initservices:
&& "${ENABLE_DAVS}" == "True" ]]; then
@ENABLED_SERVICES+=" $$service"
@fi
@if [[ "$$service" == "migrid-lustre-quota" \
@if [[ "$$service" == "migrid-quota" \
&& "${ENABLE_QUOTA}" == "True" ]]; then
@ENABLED_SERVICES+=" $$service"
@fi
Expand Down
3 changes: 3 additions & 0 deletions doc/source/sections/configuration/variables.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -635,6 +635,9 @@ Variables
* - QUOTA_BACKEND
- ""
- Filesystem backend used to set quotas and fetch data usage. Supported backends: 'lustre' and 'lustre-gocryptfs'
* - QUOTA_UPDATE_INTERVAL
- 3600
- The frequency (seconds) of storage quota updates.
* - QUOTA_USER_LIMIT
- 1099511627776
- Data limit for MiG users in bytes.
Expand Down
29 changes: 17 additions & 12 deletions docker-compose_production.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ services:
# RUN_SERVICES specifies which daemons to launch
# Available target service names follow migrid init script and are:
# httpd script monitor sshmux events cron transfers janitor
# openid sftp sftpsubsys webdavs ftps notify imnotify vmproxy
# openid sftp sftpsubsys webdavs ftps notify imnotify vmproxy quota
RUN_SERVICES:
volumes:
- type: volume
Expand Down Expand Up @@ -85,7 +85,7 @@ services:
# RUN_SERVICES specifies which daemons to launch
# Available target service names follow migrid init script and are:
# httpd script monitor sshmux events cron transfers janitor
# openid sftp sftpsubsys webdavs ftps notify imnotify vmproxy
# openid sftp sftpsubsys webdavs ftps notify imnotify vmproxy quota
RUN_SERVICES: httpd script monitor sshmux events cron transfers janitor imnotify vmproxy notify crond rsyslogd
depends_on:
# IMPORTANT: finish volume init before migrid launch as explained above
Expand Down Expand Up @@ -235,7 +235,7 @@ services:
# RUN_SERVICES specifies which daemons to launch
# Available target service names follow migrid init script and are:
# httpd script monitor sshmux events cron transfers janitor
# openid sftp sftpsubsys webdavs ftps notify imnotify vmproxy
# openid sftp sftpsubsys webdavs ftps notify imnotify vmproxy quota
RUN_SERVICES: openid rsyslogd
depends_on:
- migrid
Expand Down Expand Up @@ -380,7 +380,7 @@ services:
# RUN_SERVICES specifies which daemons to launch
# Available target service names follow migrid init script and are:
# httpd script monitor sshmux events cron transfers janitor
# openid sftp sftpsubsys webdavs ftps notify imnotify vmproxy
# openid sftp sftpsubsys webdavs ftps notify imnotify vmproxy quota
RUN_SERVICES: sftp sftpsubsys rsyslogd
depends_on:
- migrid
Expand Down Expand Up @@ -526,7 +526,7 @@ services:
# RUN_SERVICES specifies which daemons to launch
# Available target service names follow migrid init script and are:
# httpd script monitor sshmux events cron transfers janitor
# openid sftp sftpsubsys webdavs ftps notify imnotify vmproxy
# openid sftp sftpsubsys webdavs ftps notify imnotify vmproxy quota
RUN_SERVICES: ftps rsyslogd
depends_on:
- migrid
Expand Down Expand Up @@ -671,7 +671,7 @@ services:
# RUN_SERVICES specifies which daemons to launch
# Available target service names follow migrid init script and are:
# httpd script monitor sshmux events cron transfers janitor
# openid sftp sftpsubsys webdavs ftps notify imnotify vmproxy
# openid sftp sftpsubsys webdavs ftps notify imnotify vmproxy quota
RUN_SERVICES: webdavs rsyslogd
depends_on:
- migrid
Expand Down Expand Up @@ -807,12 +807,17 @@ services:
target: /home/mig/state/mig_system_run
command: /app/docker-entry.sh -k

migrid-lustre-quota:
container_name: migrid-lustre-quota
migrid-quota:
container_name: migrid-quota
image: ${CONTAINER_REGISTRY}/ucphhpc/migrid${CONTAINER_TAG}
network_mode: host
environment:
TZ: ${TZ}
# RUN_SERVICES specifies which daemons to launch
# Available target service names follow migrid init script and are:
# httpd script monitor sshmux events cron transfers janitor
# openid sftp sftpsubsys webdavs ftps notify imnotify vmproxy quota
RUN_SERVICES: quota rsyslogd
depends_on:
- migrid
cap_add:
Expand All @@ -837,7 +842,7 @@ services:
source: mig-server-extconfs
target: /home/mig/mig/server/MiGserver.d
- type: volume
source: migrid-lustre-quota-syslog
source: migrid-quota-syslog
target: /var/log
- type: volume
source: state
Expand All @@ -863,7 +868,7 @@ services:
- type: volume
source: vgrid_files_writable
target: /home/mig/state/vgrid_files_writable
command: miglustrequota.py -c /home/mig/mig/server/MiGserver.conf
command: /app/docker-entry.sh -k

# NOTE: not used in stand-alone production mode
# nginx-proxy:
Expand Down Expand Up @@ -1216,10 +1221,10 @@ volumes:
device: ${LOG_ROOT}/syslog/migrid-ftps
o: bind

migrid-lustre-quota-syslog:
migrid-quota-syslog:
# Volume used for exposing migrid lustre quota container system log
driver: local
driver_opts:
type: none
device: ${LOG_ROOT}/syslog/migrid-lustre-quota
device: ${LOG_ROOT}/syslog/migrid-quota
o: bind
3 changes: 3 additions & 0 deletions docker-entry.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -229,6 +229,9 @@ while [ ${KEEP_RUNNING} -eq 1 ]; do
elif [ "$svc" = "crond" ]; then
PROCNAME="$svc"
PROCUSER="root"
elif [ "$svc" = "quota" ]; then
PROCNAME="grid_quota"
Copy link
Contributor

@jonasbardino jonasbardino Nov 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit uncertain or surprised that this process name would need to be spelled out explicitly. Isn't it just $svc as in 'quota'? If not I'd suppose it would have e.g. a .py extension.

Copy link
Contributor Author

@Martin-Rehr Martin-Rehr Nov 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it's run as 'root' I choose to prefix it with 'grid' to avoid name clashes with other potential 'root' 'quota' processes.

'.py' could be added but is not required.

PROCUSER="root"
else
PROCNAME="$svc"
PROCUSER=$USER
Expand Down
1 change: 1 addition & 0 deletions production.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -267,6 +267,7 @@ CLOUD_SERVICES_DESC="{'MIST': '/home/mig/state/wwwpublic/mist_desc.html'}"
# NOTE: lustre >=2.15.4 is required for quota support
# supported backends: 'lustre' and 'lustre-gocryptfs'
QUOTA_BACKEND=""
QUOTA_UPDATE_INTERVAL=3600
QUOTA_USER_LIMIT=1099511627776
QUOTA_VGRID_LIMIT=1099511627776
QUOTA_LUSTRE_VERSION=2.15.4
Expand Down
Loading