Merge pull request #1663 from thunderstore-io/12-10-docker_dev_env_setup

Set up Docker development environment for Cyberstorm Remix

Author: Oksamies

Dockerfile.dev

@@ -0,0 +1,10 @@
+FROM node:24-bookworm
+
+WORKDIR /workspace
+
+# Copy entrypoint script
+COPY apps/cyberstorm-remix/entrypoint.dev.sh /usr/local/bin/entrypoint.sh
+RUN chmod +x /usr/local/bin/entrypoint.sh
+
+ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
+CMD ["yarn", "workspace", "@thunderstore/cyberstorm-remix", "dev", "--port", "3000", "--host", "0.0.0.0"]

apps/cyberstorm-remix/README.md

@@ -1,68 +1,45 @@
-# How to setup and run Nimbus dev environment with local Thunderstore
-
-## Preparations
-This quide expects you to have setup your Thunderstore Django project for development on some level. Please setup the Thunderstore project before continuing.
-
-## Setup nginx proxy for local data ingress/egress
-1. Add the following to your hosts (on windows, google what how to achive same on other OS')
-```
-127.0.0.1 thunderstore.temp
-127.0.0.1 new.thunderstore.temp
-```
-
-2. Boot up the nginx proxy with the following command; `docker compose -f tools/ts-dev-proxy/docker-compose.yml up`
-
-3. Boot up your Thunderstore backend and ensure it's running in port 81 (it's normally 80). The following [line](https://github.com/thunderstore-io/Thunderstore/blob/f06b9b438ea6e990881e60339d34bde1a480d073/docker-compose.yml#L123) in your Thunderstore projects docker-compose, should be `- "127.0.0.1:81:8000"`
-
-## Setup Nimbus for development
-
-1. Clone the repo `git@github.com:thunderstore-io/thunderstore-ui.git`
-
-2. Setup FontAwesome token. One way to do it is to add a `.npmrc` file with the following contents, under `thunderstore-ui/build-secrets/.npmrc`
-```
-@fortawesome:registry = "https://npm.fontawesome.com/"
-//npm.fontawesome.com/:_authToken = YOUR_FA_TOKEN
-```
-
-3. Run `yarn install` first on the repo root (`thunderstore-ui` folder) and then again in the `thunderstore-ui/apps/cyberstorm-remix` folder.
-
-4. Add `.env.development` and/or `.env.production` files. You can copy the `.env` file, rename and edit the values to your needs. Here's a example of the file contents:
-```
-VITE_SITE_URL=http://thunderstore.temp
-VITE_BETA_SITE_URL=http://new.thunderstore.temp
-VITE_API_URL=http://thunderstore.temp
-VITE_COOKIE_DOMAIN=.thunderstore.temp
-VITE_AUTH_BASE_URL=http://thunderstore.temp
-VITE_AUTH_RETURN_URL=http://new.thunderstore.temp
-__VITE_ADDITIONAL_SERVER_ALLOWED_HOSTS=.thunderstore.temp
-```
-
-5. Run the build/start server script or the dev server script
-
-Build and start
-```
-yarn workspace @thunderstore/cyberstorm-remix build
-yarn workspace @thunderstore/cyberstorm-remix start --port 3000 --host 0.0.0.0
-```
-
-Dev script
-```
-yarn workspace @thunderstore/cyberstorm-remix dev --port 3000 --host 0.0.0.0
-```
-
-## Finally
-You should now have the fully local Nimbus dev environment setup and running in `http://new.thunderstore.temp` and the Django site should be running in `http://thunderstore.temp`. You might need to go to `http://new.thunderstore.temp/communities` as Nimbus doesn't have a landing page yet.
-
-# How to setup ts-proxy as a backend for this project
-**Keep in mind that when using the ts-proxy, the sessions and actions will be made against the actual production Thunderstore**
-
-1. Open hosts file as administrator (`C:\Windows\System32\drivers\etc`) and add this `127.0.0.1 thunderstore.temp` there
-2. Download and install Docker and docker-compose. For windows people, Docker for Windows should be enough.
-3. Open up a terminal and navigate to `thunderstore-ui/tools/ts-proxy`
-4. Run `docker compose up`
-5. Add these to your `.env.development` or `.env.production`
-```
-PUBLIC_SITE_URL=http://thunderstore.temp
-PUBLIC_API_URL=http://thunderstore.temp:81
-```
-6. Run the Nimbus project normally
+# Cyberstorm Remix (Nimbus)
+
+This is the Remix application that powers the new Thunderstore frontend (codenamed Nimbus).
+
+## Quick Start (Docker)
+
+The easiest way to run the full stack (Backend + Frontend) is using Docker.
+
+1.  **Clone Repositories**
+    Ensure you have both `Thunderstore` (Backend) and `thunderstore-ui` (Frontend) cloned side-by-side.
+    ```bash
+    # Example structure
+    # C:\projects\Thunderstore
+    # C:\projects\thunderstore-ui
+    ```
+
+2.  **Start Backend**
+    ```bash
+    cd ../Thunderstore
+    docker compose up -d
+    docker compose exec django python manage.py setup_dev_env
+    ```
+    (If you have some pre-existing containers, please do `docker compose down -v`, `docker compose up -d --build` and `docker compose exec django python manage.py setup_dev_env`)
+
+3.  **Start Frontend**
+    ```bash
+    cd ../thunderstore-ui
+    docker compose -f docker-compose.remix.development.yml up -d
+    ```
+    (If you have some pre-existing containers, please do `docker compose -f docker-compose.remix.development.yml down -v` and `docker compose -f docker-compose.remix.development.yml up -d --build`)
+
+4.  **Open Browser**
+    -   **Frontend**: [http://new.localhost](http://new.localhost)
+    -   **Backend**: [http://localhost](http://localhost)
+
+## Manual Setup
+
+If you prefer running Node locally instead of in Docker:
+
+1.  **Install Dependencies**: `yarn install` (in repo root)
+2.  **Configure Env**: Copy `.env.example` to `.env` in `apps/cyberstorm-remix`.
+3.  **Run Dev Server**:
+    ```bash
+    yarn workspace @thunderstore/cyberstorm-remix dev --port 3000 --host 0.0.0.0
+    ```

apps/cyberstorm-remix/entrypoint.dev.sh

@@ -0,0 +1,57 @@
+#!/bin/sh
+set -e
+
+# Helper to ensure dirs exist and are writable by the node user.
+ensure_node_dirs() {
+  for dir in "$@"; do
+    mkdir -p "$dir"
+    chown node:node "$dir"
+  done
+}
+
+# Sync Nginx configs
+mkdir -p /etc/nginx/user-conf
+cp /workspace/tools/nginx/*.conf /etc/nginx/user-conf/ 2>/dev/null || true
+
+# Setup npmrc
+if [ -f /run/secrets/npmrc ]; then
+  # Copy with tight permissions; file contains auth tokens.
+  install -m 0600 -o node -g node /run/secrets/npmrc /home/node/.npmrc
+fi
+
+ensure_node_dirs \
+  node_modules \
+  apps/cyberstorm-remix/node_modules \
+  apps/cyberstorm-remix/.react-router
+
+# Install dependencies if missing or if react-router is missing
+if [ -z "$(ls -A node_modules)" ] || [ ! -f node_modules/.bin/react-router ]; then
+    echo "Installing dependencies..."
+    # Fix permissions recursively before install to ensure we can write.
+    chown -R node:node node_modules apps/cyberstorm-remix/node_modules
+
+    su node -c "yarn install --frozen-lockfile --production=false"
+fi
+
+# Map localhost to the backend nginx container IP so SSR API calls hit the backend
+# without changing client-side env values.
+backend_nginx_ip=$(getent hosts nginx | awk '{print $1}')
+if [ -n "$backend_nginx_ip" ]; then
+  # Prepend to /etc/hosts to take precedence over the default loopback entry.
+  # Make idempotent to avoid duplicate lines on restarts.
+  if ! grep -q "^${backend_nginx_ip}[[:space:]][[:space:]]*localhost\([[:space:]]\|$\)" /etc/hosts 2>/dev/null; then
+    tmp_hosts=$(mktemp)
+    {
+      printf "%s localhost\n" "$backend_nginx_ip"
+      # Drop existing localhost mappings so the new mapping is unambiguous.
+      # This keeps other host entries intact.
+      awk '!($0 ~ /(^|[[:space:]])localhost([[:space:]]|$)/)' /etc/hosts 2>/dev/null || true
+    } > "$tmp_hosts"
+    cat "$tmp_hosts" > /etc/hosts || printf "%s localhost\n" "$backend_nginx_ip" >> /etc/hosts
+    rm -f "$tmp_hosts"
+  fi
+fi
+
+# Execute the passed command as node (preserve argument boundaries)
+# `su -c` uses the first arg after `--` as $0, so provide a dummy.
+exec su node -c 'exec "$@"' -- _ "$@"

docker-compose.remix.development.yml

@@ -1,46 +1,73 @@
-version: "3.8"
+## Shared config for local file sync containers
+x-rsync-common: &rsync-common
+  image: alpine:3.20
+  volumes:
+    - ./:/src:ro
+    - workspace_src:/workspace
+    - thunderstore_nginx_conf:/etc/nginx/user-conf
+  environment:
+    RSYNC_ARGS: "-a --delete --info=progress2 --exclude=.git --exclude=build-secrets --exclude=.npmrc --exclude=node_modules --exclude=.turbo --exclude=.cache --exclude=apps/cyberstorm-remix/build --exclude=apps/cyberstorm-remix/.react-router"
+    SYNC_INTERVAL: "5"
 
 services:
+  cyberstorm-remix-sync:
+    <<: *rsync-common
+    container_name: cyberstorm-remix-sync
+    restart: "no"
+    command: >-
+      /bin/sh -c "set -e; apk add --no-cache rsync; rsync $$RSYNC_ARGS /src/ /workspace/; tmp=/etc/nginx/user-conf/.new-localhost.conf.tmp; cp -f /src/tools/nginx/new-localhost.conf \"$$tmp\"; mv -f \"$$tmp\" /etc/nginx/user-conf/new-localhost.conf; echo sync complete"
+
+  cyberstorm-remix-watch:
+    <<: *rsync-common
+    container_name: cyberstorm-remix-watch
+    restart: unless-stopped
+    depends_on:
+      cyberstorm-remix-sync:
+        condition: service_completed_successfully
+    command: >-
+      /bin/sh -c "set -e; apk add --no-cache rsync; while true; do rsync $$RSYNC_ARGS /src/ /workspace/; tmp=/etc/nginx/user-conf/.new-localhost.conf.tmp; cp -f /src/tools/nginx/new-localhost.conf \"$$tmp\"; mv -f \"$$tmp\" /etc/nginx/user-conf/new-localhost.conf; sleep $$SYNC_INTERVAL; done"
+
   cyberstorm-remix:
     container_name: cyberstorm-remix
-    command: /bin/sh -c "tail -f /dev/null"
-    # command: yarn workspace @thunderstore/cyberstorm-remix dev --port 3000 --host 0.0.0.0
     build:
-      context: "./"
-      dockerfile: "apps/cyberstorm-remix/Dockerfile.development"
-      secrets:
-        - "npmrc"
-    image: thunderstore/thunderstore-ui.cyberstorm-remix-development:${IMAGE_TAG:-dev}
+      context: ./
+      dockerfile: ./Dockerfile.dev
+    working_dir: /workspace
     ports:
       - "127.0.0.1:3000:3000"
+    depends_on:
+      cyberstorm-remix-sync:
+        condition: service_completed_successfully
     volumes:
-      - remix_node_modules:/app/apps/cyberstorm-remix/node_modules
-      - ./apps/cyberstorm-remix:/app/apps/cyberstorm-remix
-      - ./packages:/app/packages
+      - workspace_src:/workspace
+      - thunderstore_ui_node_modules:/workspace/node_modules
+      - nimbus_node_modules:/workspace/apps/cyberstorm-remix/node_modules
+      - thunderstore_nginx_conf:/etc/nginx/user-conf
+      - yarn_cache:/usr/local/share/.cache/yarn
     networks:
         - thunderstore_default
     environment:
       - NODE_ENV=development
-
-  remix-nginx-proxy:
-    image: nginx
-    container_name: remix-nginx-proxy
-    restart: always
-    ports:
-      - "443:443"
-      - "80:80"
-    volumes:
-      - ./nginx/www:/var/www:ro
-      - ./nginx/conf.d/remixdefault.conf:/etc/nginx/conf.d/default.conf:ro
-      # - ./ssl:/etc/nginx/ssl:ro
-      # - ./snippets:/etc/nginx/snippets:ro
-      # - ./protect:/etc/nginx/protect:ro
-    hostname: thunderstore.temp
-    networks:
-        - thunderstore_default
+      - NPM_CONFIG_USERCONFIG=/run/secrets/npmrc
+      - ENABLE_BROKEN_PAGES=True
+      - VITE_DEVELOPMENT=True
+      - VITE_SITE_URL=http://localhost
+      - VITE_BETA_SITE_URL=http://new.localhost
+      - VITE_API_URL=http://localhost
+      - VITE_COOKIE_DOMAIN=.localhost
+      - VITE_AUTH_BASE_URL=http://localhost
+      - VITE_AUTH_RETURN_URL=http://new.localhost
+      - __VITE_ADDITIONAL_SERVER_ALLOWED_HOSTS=.localhost
+    secrets:
+      - npmrc
 
 volumes:
-  remix_node_modules:
+  workspace_src:
+  thunderstore_ui_node_modules:
+  nimbus_node_modules:
+  yarn_cache:
+  thunderstore_nginx_conf:
+    external: true
 
 secrets:
   npmrc:

tools/nginx/new-localhost.conf

@@ -0,0 +1,24 @@
+map $http_upgrade $connection_upgrade {
+    default upgrade;
+    '' '';
+}
+
+server {
+    listen 80;
+    server_name new.localhost;
+
+    # Resolve container DNS dynamically to avoid stale IPs.
+    resolver 127.0.0.11 ipv6=off valid=30s;
+
+    location / {
+        # Ensure requests to new.localhost are never served from the backend nginx proxy cache.
+        proxy_cache off;
+        proxy_cache_bypass 1;
+        proxy_no_cache 1;
+        proxy_pass http://cyberstorm-remix:3000;
+        proxy_http_version 1.1;
+        proxy_set_header Host $host;
+        proxy_set_header Upgrade $http_upgrade;
+        proxy_set_header Connection $connection_upgrade;
+    }
+}