46 updated use GitHub codespaces #103

kobebuckley · 2024-06-05T22:46:22Z

Start a Fresh Codespace ( ~ 5 minutes )

Open a new branch from an issue
Click on the 'Code' Dropdown arrow
Click on the '...' to the right of 'Codespaces'
Select 'New with Options'
Select the proper branch for the issue you will work on > region that is closest to you > then choose '8 core with 32 gb'
Create Codespace & wait for everything to load ( around 5 minutes )
A terminal should open that says ' Finishing up > Running postCreateCommand > and Running PostStartCommand'
Once the 'PostStartCommand' has a Green checkmark > we then follow the next steps

Run the Codespace ( ~ 15 minutes )

Steps based on the 'Quick Start' guide from the 'Getting Started' documentation
https://github.com/CodeWithAloha/uipa/blob/main/docs/Getting-Started.md

In a terminal window, run docker ps. (if this is not running, then run docker-compose up)
In another terminal window, yarn build.
In another terminal window, run bash data/seed/init_db.sh.
Enter your email address ( or random fake email ) > Enter a username > Enter at least a 9-character password
In that same terminal window, run python manage.py runserver.
Click on the 'Open in Browser' prompt on the bottom right of your codespace for http://127.0.0.1:8000/ .

tyliec · 2024-06-09T20:53:16Z

Dockerfile

+RUN /workspace/venv/bin/pip install --upgrade pip
+
+# Install dependencies in smaller groups to identify the problematic package
+RUN /workspace/venv/bin/pip install \


Should we convert this back to installing the requirements.txt file?

tyliec · 2024-06-09T20:53:47Z

install_docker.sh

Is this file being used somewhere?

tyliec · 2024-06-09T20:53:58Z

get-docker.sh

Is this file being used somewhere?

tyliec · 2024-06-09T20:55:36Z

.devcontainer/devcontainer.json

+	"forwardPorts": [8000, 5432, 9200],
+	// "postCreateCommand": "sudo usermod -aG docker $USER && sudo service docker start",
+	"postCreateCommand": "sudo usermod -aG docker $USER",
+	"postStartCommand": "bash cleanup_and_start.sh",


Did you find that we needed to do this sort of cleanup and starting of Docker for some reason?

I started a Codespace off the main branch and did a docker info - Docker should be installed and run by default.

@tyliec ➜ /workspaces/uipa (main) $ docker info Client: Version: 26.1.3-1 Context: default Debug Mode: false Plugins: buildx: Docker Buildx (Docker Inc.) Version: v0.14.1 Path: /usr/libexec/docker/cli-plugins/docker-buildx compose: Docker Compose (Docker Inc.) Version: v2.27.1 Path: /usr/libexec/docker/cli-plugins/docker-compose ...

Not sure if you found some sort of timing issue or not here though

tyliec · 2024-06-09T21:02:18Z

Dockerfile

+FROM mcr.microsoft.com/devcontainers/javascript-node:22
+
+# Install main dependencies and Docker in one step to reduce layers
+RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \


Do we actually need all of these packages? Codespaces should have a lot of these by default already - i.e. git, cmake, ...etc.

From a new Codespace:

@tyliec ➜ /workspaces/uipa (main) $ dpkg -l cmake Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-==============-=======================-============-======================================= ii cmake 3.16.3-1ubuntu1.20.04.1 amd64 cross-platform, open-source make system

tyliec · 2024-06-09T21:03:29Z

Dockerfile

@@ -0,0 +1,123 @@
+# FROM mcr.microsoft.com/devcontainers/javascript-node:1-18
+FROM mcr.microsoft.com/devcontainers/javascript-node:22


Any particular reason you chose this base image? Given this is a Python / Django project primarily, I would take a look at https://github.com/devcontainers/images/tree/main/src/python instead

tyliec

We should talk about this PR - but I would take a closer look at https://github.com/devcontainers/templates/tree/main/src/python for a few examples of implementation, and maybe https://containers.dev/ for a general idea of the specification

tyliec · 2024-06-09T21:14:08Z

.devcontainer/devcontainer.json

+	  "dockerfile": "../Dockerfile"
+	},
+	"runArgs": [
+	  "--init",


Is there a specific reason we chose to user a Docker in Docker approach here?

I think the DinD (Docker in Docker) approach is used to allow starting up the database and search engine from a terminal in the dev container. The dev container's postStartCommand property says to run the cleanup_and_start.sh script. It in turn runs docker-compose to start the dev web server, database, and search engine in containers.

The way I read the Dev Containers docs, I'm guessing that using a docker-compose.yml file in the build property of the devcontainer.json file, the database and search engine could be run automatically along side the dev web server without having to run them all from inside a dev container.

In the existing docker-compose.yml file, if you rename the newly added web service to devcontainer that will be the main dev container and the db and elasticsearch services will be additional containers accessible by the dev container. See https://containers.dev/guide/dockerfile#docker-compose. Of course, other changes would be required to have things play nicely.

A doc about how use a dev container for new people (possible contributors) to help them get up and running quickly should be included. It should say what this dev container set-up provides for a developer or person who wants to run the website.

Also, if I'm not mistaken, if we set up the dev container to use docker-compose.yml then it would also be possible for people to use it locally without running a dev container. While the free GitHub personal account provides a limited amount of Codespaces resources to run dev containers, it is better not to be required to use Codespaces as it may incur costs that people aren't willing to pay.

russtoku · 2024-06-12T07:02:20Z

Opening the 46-Update-use-github-codespaces branch in a codespace results in no joy. I can open a terminal but python manage.py check results in an error:

(venv) @russtoku ➜ /workspaces/uipa (46-Updated-use-github-codespaces) $ python manage.py check
/workspaces/uipa/uipa_org/settings/base.py:105: SyntaxWarning: invalid escape sequence '\.'
  greetings=[rec(u"Aloha (?:Mr\.?|Ms\.? .*?)")],
/workspace/venv/lib/python3.12/site-packages/floppyforms/__init__.py:20: UserWarning: Unable to import floppyforms.gis, geometry widgets not available
  warnings.warn(
Traceback (most recent call last):
  File "/workspaces/uipa/manage.py", line 13, in <module>
    execute_from_command_line(sys.argv)
  File "/workspace/venv/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/workspace/venv/lib/python3.12/site-packages/django/core/management/__init__.py", line 416, in execute
    django.setup()
  File "/workspace/venv/lib/python3.12/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/workspace/venv/lib/python3.12/site-packages/django/apps/registry.py", line 116, in populate
    app_config.import_models()
  File "/workspace/venv/lib/python3.12/site-packages/django/apps/config.py", line 269, in import_models
    self.models_module = import_module(models_module_name)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/workspace/venv/lib/python3.12/site-packages/django/contrib/auth/models.py", line 3, in <module>
    from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
  File "/workspace/venv/lib/python3.12/site-packages/django/contrib/auth/base_user.py", line 57, in <module>
    class AbstractBaseUser(models.Model):
  File "/workspace/venv/lib/python3.12/site-packages/django/db/models/base.py", line 143, in __new__
    new_class.add_to_class("_meta", Options(meta, app_label))
  File "/workspace/venv/lib/python3.12/site-packages/django/db/models/base.py", line 371, in add_to_class
    value.contribute_to_class(cls, name)
  File "/workspace/venv/lib/python3.12/site-packages/django/db/models/options.py", line 243, in contribute_to_class
    self.db_table, connection.ops.max_name_length()
                   ^^^^^^^^^^^^^^
  File "/workspace/venv/lib/python3.12/site-packages/django/utils/connection.py", line 15, in __getattr__
    return getattr(self._connections[self._alias], item)
                   ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/workspace/venv/lib/python3.12/site-packages/django/utils/connection.py", line 62, in __getitem__
    conn = self.create_connection(alias)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/venv/lib/python3.12/site-packages/django/db/utils.py", line 193, in create_connection
    backend = load_backend(db["ENGINE"])
              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/venv/lib/python3.12/site-packages/django/db/utils.py", line 113, in load_backend
    return import_module("%s.base" % backend_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/venv/lib/python3.12/site-packages/django/contrib/gis/db/backends/postgis/base.py", line 7, in <module>
    from .adapter import PostGISAdapter
  File "/workspace/venv/lib/python3.12/site-packages/django/contrib/gis/db/backends/postgis/adapter.py", line 5, in <module>
    from django.contrib.gis.geos import GEOSGeometry
  File "/workspace/venv/lib/python3.12/site-packages/django/contrib/gis/geos/__init__.py", line 5, in <module>
    from .collections import (  # NOQA
  File "/workspace/venv/lib/python3.12/site-packages/django/contrib/gis/geos/collections.py", line 6, in <module>
    from django.contrib.gis.geos.geometry import GEOSGeometry, LinearGeometryMixin
  File "/workspace/venv/lib/python3.12/site-packages/django/contrib/gis/geos/geometry.py", line 8, in <module>
    from django.contrib.gis import gdal
  File "/workspace/venv/lib/python3.12/site-packages/django/contrib/gis/gdal/__init__.py", line 28, in <module>
    from django.contrib.gis.gdal.datasource import DataSource
  File "/workspace/venv/lib/python3.12/site-packages/django/contrib/gis/gdal/datasource.py", line 40, in <module>
    from django.contrib.gis.gdal.driver import Driver
  File "/workspace/venv/lib/python3.12/site-packages/django/contrib/gis/gdal/driver.py", line 5, in <module>
    from django.contrib.gis.gdal.prototypes import ds as vcapi
  File "/workspace/venv/lib/python3.12/site-packages/django/contrib/gis/gdal/prototypes/ds.py", line 9, in <module>
    from django.contrib.gis.gdal.libgdal import lgdal
  File "/workspace/venv/lib/python3.12/site-packages/django/contrib/gis/gdal/libgdal.py", line 64, in <module>
    raise ImproperlyConfigured(
django.core.exceptions.ImproperlyConfigured: Could not find the GDAL library (tried "gdal", "GDAL", "gdal3.6.0", "gdal3.5.0", "gdal3.4.0", "gdal3.3.0", "gdal3.2.0", "gdal3.1.0", "gdal3.0.0", "gdal2.4.0", "gdal2.3.0", "gdal2.2.0"). Is GDAL installed? If it is, try setting GDAL_LIBRARY_PATH in your settings.

There are libraries that are missing in the dev container.

The database and search engine are not accessible from the dev container.

russtoku · 2024-06-12T17:53:26Z

What state do we want the dev container to be in after it's been started up and attached to by the GitHub web interface?

I think that there are two audiences with different needs:

Developer (frontend, backend, and full-stack)
Non-developer (user testing, documentation writer, etc.)

The non-developer would like to be able to go to the web app and start playing with it; not open a terminal window and type commands in.

The developer should be OK with opening a terminal window and typing in commands.

I think that both can be accomodated.

It's very important that a document that provides guidance be provided so that new contributors can come up to speed quickly and easily.

tyliec · 2024-06-13T05:24:24Z

Dockerfile

+RUN pip install -r requirements.txt
+
+# Start Docker daemon
+CMD ["sudo", "service", "docker", "start"]


This should be sleep infinity, or this should be specified as sleep infinity in the devcontainer.json

… up sudo docker-compose up --build

…_db.sh

…s not passing health check

… an error with django-configurations

…o attribute 'geo_db_type'

…tainers

…sult: Confirmed that both are accessible with: sudo docker exec -it devcontainer_app_1 /bin/bash psql --version curl --version

… has no attribute 'geo_db_type'

kobebuckley requested a review from a team as a code owner June 5, 2024 22:46

kobebuckley linked an issue Jun 5, 2024 that may be closed by this pull request

Use GitHub Codespaces #46

Open

tyliec reviewed Jun 9, 2024

View reviewed changes

install_docker.sh Outdated

Copy link

Member

tyliec Jun 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this file being used somewhere?

tyliec reviewed Jun 9, 2024

View reviewed changes

get-docker.sh Outdated

Copy link

Member

tyliec Jun 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this file being used somewhere?

tyliec reviewed Jun 9, 2024

View reviewed changes

tyliec requested changes Jun 9, 2024

View reviewed changes

tyliec reviewed Jun 9, 2024

View reviewed changes

tyliec reviewed Jun 13, 2024

View reviewed changes

tyliec added 4 commits June 17, 2024 18:18

.devcontainer

c150d74

remove remoteUser override

3bb9f7f

testing

c5dbb77

test again

185e2f2

kobebuckley marked this pull request as draft June 18, 2024 04:51

tyliec and others added 2 commits June 17, 2024 18:55

test

f7ef992

working basics

e7a1025

kobebuckley force-pushed the 46-Updated-use-github-codespaces branch 2 times, most recently from 180daca to 4950ede Compare June 22, 2024 05:47

kobebuckley added 2 commits July 2, 2024 20:19

base working + docker compose up functions locally

adbe771

locally installed requirements through dockerfile - no error spinning…

ccc3fcc

… up sudo docker-compose up --build

kobebuckley force-pushed the 46-Updated-use-github-codespaces branch from b7485d2 to ccc3fcc Compare July 3, 2024 06:34

kobebuckley added 5 commits July 2, 2024 21:04

functional docker-compose up, next is testing the bash data/seed/init…

00a2391

…_db.sh

trying to fix port 8000, before it would run but kept closing

3caba9b

further testing needing for automation of server setup

67d8c12

copied entire root dir into docker for testing, 1 of the containers i…

c759734

…s not passing health check

still unhealthy with extra timer, will need more investigation

6af8a79

kobebuckley added 9 commits July 21, 2024 21:40

updated health check to provide creds using ' -u '

7c81652

changes to compose to connect postgres and elastic to app 0 currently…

a424f34

… an error with django-configurations

working on the #! | AttributeError: 'DatabaseOperations' object has n…

741048c

…o attribute 'geo_db_type'

changes to get the static files error and to ensure readyness for con…

b8d50bf

…tainers

app trying to connect to DB, but unable to establish a connection

ee54264

installed postgresql-client and curl onto dockerfile dependencies, re…

f799ff3

…sult: Confirmed that both are accessible with: sudo docker exec -it devcontainer_app_1 /bin/bash psql --version curl --version

added a password for base user to test out pings and other installs

f51e839

missing geo_db_type

1b2b47a

still issues with app_1 | AttributeError: 'DatabaseOperations' object…

30eb9b7

… has no attribute 'geo_db_type'

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

46 updated use GitHub codespaces #103

46 updated use GitHub codespaces #103

kobebuckley commented Jun 5, 2024

tyliec Jun 9, 2024

tyliec Jun 9, 2024

tyliec Jun 9, 2024

tyliec Jun 9, 2024

tyliec Jun 9, 2024

tyliec Jun 9, 2024

tyliec Jun 9, 2024

tyliec left a comment

tyliec Jun 9, 2024

russtoku Jun 11, 2024 •

edited

Loading

russtoku commented Jun 12, 2024

russtoku commented Jun 12, 2024

tyliec Jun 13, 2024

		@@ -0,0 +1,123 @@
		# FROM mcr.microsoft.com/devcontainers/javascript-node:1-18
		FROM mcr.microsoft.com/devcontainers/javascript-node:22

46 updated use GitHub codespaces #103

Are you sure you want to change the base?

46 updated use GitHub codespaces #103

Conversation

kobebuckley commented Jun 5, 2024

Start a Fresh Codespace ( ~ 5 minutes )

Run the Codespace ( ~ 15 minutes )

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tyliec left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

russtoku Jun 11, 2024 • edited Loading

Choose a reason for hiding this comment

russtoku commented Jun 12, 2024

russtoku commented Jun 12, 2024

Choose a reason for hiding this comment

russtoku Jun 11, 2024 •

edited

Loading