In the current state of the application, development is meant to be done running the application in docker container. It is not the fastest (specially under MacOS), and sometimes most convinient choice in development but it simplyfies working with the various side services like keycloak, mariadb, redis etc.
You need the following installed on your system to run this application in development mode:
- Ruby "3.3.8". We suggest to use rbenv for managing (
rbenv install 3.3.8) the various ruby versions — 💡 Use this PPA to install it on older versions of Ubuntu Linux; - Docker with the so-called compose version 2
- there is a switch to flip in Docker Desktop for Mac for this.
- The project assumes a working
docker composecommand. The old (Python-based)docker-composemight still work.
- Common dev traefik configuration. See instructions therein.
- A secrets file defining the following environment variables (see
.env.samplefor more details):ATELA_KEY,CAMIPRO_PHOTO_KEY,DEV_ENTRA_TENANT_ID,DEV_ENTRA_CLIENT_ID,DEV_ENTRA_SECRET,EPFLAPI_PASSWORD,OPENAI_API_KEY,ORACOURS_PWD,TRANS_USER,TRANS_PASS - NVM and node version 18.x.x (
nvm install 18) andyarnneeded for building EPFL elements.
The application secrets are supposed to be accessible as env. variables set by
$KBPATH/$SECRETS, a bash script that is normally located in the project's
keybase directory $KBPATH=/keybase/team/epfl_people.prod. Off-course this is
just a path and could be anything else.
Few other variables needed for the development environment are set in the local
.env. Copy and modify the provided .env.sample file before starting anything.
Most of the default values should be ok. Therefore, there should be not much do
modify.
In order to run ./bin/rails commands directly from the console instead of docker,
secrets must be loaded into env variables with, _e.g., the following command:
. ./.env ; cat .env $KBPATH/$SECRETS | awk '/^[A-Z]/{print "export ", $0;}' | source /dev/stdinThe mostly used commands are wrapped as rules in the makefile which instanciates
all the required env variables. See make help for a full listing of the
available shortcuts.
For the application to run we need to initialize several databases because it still includes the code for migrating from the legacy application which requires various databases to work.
- restore the relevant parts of the legacy databases:
make restore. For this to work you need to have access to the legacy production server or to a server that is allowed to access the variousdinfodatabases. Setup apeo11endpoint in your~/.ssh/configfile for the import script to ssh into and dump the databases. Ideally, we should prepare a set of fake data instead but it is quite cumbersome. - migrate and seed the local application database:
make seed - if for some reason you prefer to use the locak keycloak server:
make rekc - if you are working on the importation of legacy profiles, then you need to
prepare the data with
make legaimportwhich will load all text entries from the legacy user profiles and try to guess their languages using AI
The way of debugging is through the debug gem. If you have the inner strength
to scrut its inscrutable documentation, then more
power to you.
- Put
debuggerin your source code where you want the debugger to break - run
make debugwhich will attach your terminal to the rails console where the debugger will be visible.
It would be nice to findout how to make the following work with rails running
on the container instead of locally as /bin/dev but I never found the energies
and the motivation (for me the cli is more than enough):
- Create a
.rdbgrcfile in your home directory¹ that contains a single line:open chrome
- Put
debuggerin your source code where you want the debugger to break - Run or re-run the development server as usual (i.e.
./bin/dev)
💡 The hot-reload feature doesn't work in Chrome (yet), which will continue to display the old source code. You will need to stop and restart the server (which brings Chrome down and back up again as well) to fix that.
¹ What about Windows®, you ask?... Are you sure you are a real developer?
This is the public directory of people working and studying at the EPFL. It is a complete rewrite of a legacy application written in perl that have loyally and solidly served the EPFL for many years.
The application collects personal data from various internal sources and display it in customizable profile pages. The data is also served to other services within the school. Notably, it provides people listings to the the several instances of wordpress that make up the official EPFL website.
The two main external data sources are the following web services:
api.epfl.chfor all official informations (name, status, positions);isa.epfl.chfor academic information regardin teachers: courses, PhD students etc.
The information taken from the official sources, is integrated with information optionally provided by the people and kept on the internal database.
The current implementation which tries to closely mimic its legacy ancestor. In particular, we tried to keep the idea of a profile page composed of several content blocks. We have tried to implement the blocks in a way that will allow new powerfull blocks to be added in the future. For the moment, we have just two types of blocks: rich text boxes that are just a title with a free content in html format, and index boxes that are containers of lists of records of the same model. Presently we have implemented models for listing the person's studies, work experiences, awards, selected publications, and links to various research or social network websites.
See this comic to find out why using one of these newfangled NoSQL data stores might not be the best idea for a business-oriented application.
When it comes to modeling (as in the M of MVC) data into a relational database, Rails' ORM is tough to beat. For instance, Red Hat has an entire section of its business strategy which consists of writing and selling Rails front-ends to neckbeard-oriented systems — to wit: OKD for Kubernetes; Foreman for that whole IPMI / PXE / DHCP / TFTP / DNS hairball; and many more. Only occasionally will they use Django instead (e.g. Ansible Tower, possibly because Ansible itself is written in Python).
The standard layout of EPFL. References:
This is meant to be served redy to use from an internal server. On the other hand, we need few files from its source for being able to produce a custom css stylesheet. Therefore, the project will be cloned in a user-defined directory.
The starter-kit app comes bundled with Keycloak-in-a-container, configured “as-code” (see keycloak/README.md for details). While Java is admittedly a debatable choice (even moreso for production), Keycloak is an OpenID implementation that comes complete with a GUI that will let you set up test users, groups and roles as you please. This provides a so-called hermetic developer experience: you can hack while riding the bus, and worry about integration with your “real” corporate OIDC impementation (or SAML, bridged with e.g. SATOSA) at deployment time.
Now the default is to use the new EntraID authentication system even for development as it is pubblicly available and does not restrict clients based on their IPs. Therefore, we might get rid of the keycloack part.
No longer relevant as we got rid of all dependencies on external oracle DBs but
we keep this for future memory just in case we need to revamp it.
Oracle connector is quite cumbersome to install because it needs official binaries
from oracle. For linux, see the Dockerfile. For osx, use brew to install the
oracle client as explained here:
brew tap InstantClientTap/instantclient
brew install instantclient-basic
brew install instantclient-sdk
brew install instantclient-sqlplus
gem install ruby-oci8
Common variables:
RAILS_ENV: standardREDIS_CACHE: the url of the redis server for storing cache (RoR defaults to local memory storage)CAMIPRO_PHOTO_HOST: the server for camipro profile photosENABLE_API_CACHE: enable caching of call to external api servers (api, atela, etc.)
Common secrets:
CAMIPRO_PHOTO_KEY: secret key for accessing the camipro photos serverORACOURS_PWD: ${ORACOURS_PWD} password for the orable database containing the ISA coursesATELA_KEY: ${ATELA_KEY} secret key for accessingatela.epfl.chEPFLAPI_PASSWORD: ${EPFLAPI_PASSWORD} password forapi.epfl.ch
Development only variables:
RAILS_DEVELOPMENT_HOSTS: normally only localhost is considered a dev host. Using traefik we need to add the hosts that are actually used for Rails not to complain about security.
If you get the following error message in the app console: ERROR -- omniauth: (oidc) Authentication failure! Not Found: OpenIDConnect::Discovery::DiscoveryFailed, Not Found
then you probably nuked the keycloak server and forgot to provision it with
authentication data. In this case, make kconfig should do the job.
In the out-of-the-box configuration for this demo app, only the /graphql URL is protected by OpenID access control. We posit that this is, in fact, a reasonable approach to security; and that you might want to consider designing your app so that there is no need for additional protection.
GraphQL provides for all your data access and mutation needs. It is pretty straightforward to enforce the security policy (for both access control and auditing) by checking for a so-called OpenID “claim” that is mapped to a role directly from within the relevant GraphQL controllers. The rest of your app should not disclose information (except information intended for public use) at any other endpoint; nor should it permit any mutation except, over GraphQL. In other words, you should refrain from using “traditional” Rails controllers and Web templates (either Web 1.0-style with application/x-www-form-urlencoded POSTs; or “modern” REST-style APIs with other HTTP verbs), except to serve “traditional” Web content (using HTTP GET) to unauthenticated users (such as search engines). Examples of concerns that you will be able to disregard entirely are XSRF tokens (and the secret management headaches they entail when deploying a load-balanced Rails app), ad-hoc signaling and UX, and more.
In my opinion (giova), the development overhead introduced by the so called web 2.0 is justified only in two cases:
- when the volumes are huge (e.g. facebook) and it it is less expensive to delegate as much computation as possible to the client;
- when one tries to emulate a desktop application that requires a lot of reactivity and real time rendering of the UI (e.g. google docs);
Our tiny application people.epfl.ch serves at most few requests per second and is a read-only application for most of the data. The user editable part is quite limited and simple. Therefore, it does not match any of the above use cases. The amount of nice features provided natively by RoR that would have to be discarded for embracing the web 2.0 is not justified at all.
Current application offers two options for the profile picture:
- use remote camipro image (actually locally cached version of it);
- use one of the locally uploaded images;
The GUI for selecting the image must be composed of three parts:
- toggle if picture should be visible or not;
- toggle if camipro picture is to be used (currently camipro photo is used if common.photo_ext is not 1);
- list selector for the uploaded images (currently this is decided by common.photo_ts)
Current version only support two languages: french and english. We start by doing the same with the idea of adding more languages in the future. The problem is how to make the UI usable.
For two languages we decided to have each field repeated (e.g. instead of title, we have title_en, and title_fr) and statically visible in editing forms. Forms grow fat but the user gets immediate feedback about missing translations. This approach could be extended to 3, possibly 4 languages but for sure not more than 4.
More details in Docs/multilanguage.md.
In any case, a backoffice translation service should be deployed with a validation/scoring system similar to the one we did for jilion.
- rails guides of course!
- rails design patterns
- Arel doc and examples of usage: one, two
- Logging best practices