What is the workflow for initiating collaboration?

[alandsidel]

Let me start off by saying I am an experienced developer myself (over 25 years, about 10 with git, svn, vss, etc before that), so any who answer should not be afraid to be technical! I am evaluating this addin for a situation where some non-technical users will be collaborating on "development" in a shared Access DB.

I have looked through the docs and done some experimentation, but I would just like a quick walkthrough on how this should be setup on both machines from nearly scratch. Assume a "normal" development situation for collaboration as follows:

1. Two different workstations (VMs) A and B with all required applications installed.
2. An existing, empty github repo.
3. Machine A has the database already on it and needs to get it exported, committed, and pushed.
4. Machine B needs to clone the repo and then create the DB from it, after step 3.

The first two steps are no problem. Step 2 proved a little problematic because exporting the database with the plugin caused it to create a new directory rather than asking where to put the files, which of course meant they were not put in the directory already initialized and setup with the remote repo. I've since found the "export folder" setting which I can presumably use to remedy this.

Step 4 was even more problematic. Clearly I did something wrong here.

After cloning the project, I couldn't find an easy way to recreate the database from it. The ribbon hiding the "add-ins" dropdown is, of course, unavailable until you already have a database file opened. So I created a blank database just to run the addin import and it was apparently successful, but again I was caught off guard by the fact that it just created it in the same path and with the same name as it had on machine A, which was on the user's desktop.

This strikes me as a "major" problem, because that path may not be accessible during real usage. It only existed and was accessible in this case because the account I am using on both VMs is the same. If I'd been logged on machine A as Alice and machine B as Bob, machine B would have failed to create C:\Users\Alice\.... and I'm not sure how the plugin would have coped.

In any case, after the import, the repo showed some changes and the database was empty. No forms, no queries, no data, no nothing. The vcs-index.json file is changed, and the local version has an entirely empty Components object, with the ExportDate and FullExportDate fields both changed to null.

Assistance is appreciated!

[joyfullservice]

Hi Allen,

I think I am probably long overdue for creating a video tutorial that can walk new users through this exact scenario. 😄 This is precisely what the add-in is designed to do, to streamline the process of multiple developers working on the same database project and collaborating through a VCS repository.

Here is the process I would recommend for the most basic out-of-the-box usage with minimal setup and configuration. (Actually quite similar to what you described above.)

Initial Setup

1. Create a GitHub repository for the development database.
2. Place the development database in the root of the git repository. (Don't commit it yet)
3. Open the database, and use the add-in to export the source using the default settings.
4. Commit the change to the repository. This is your initial version.

By default, it will export the source to a subfolder in the same location as the database. It will also add the default .gitignore and .gitattributes files to the repository, which will do some helpful things like excluding the binary database file from the repository.

Client Setup (Other Developers)

1. Clone the GitHub repository. (This will pull down the source files, but not the original binary database file.)
2. Use the add-in to build the database from source, selecting the source folder in the repository. (i.e. \Database.accdb.src\) If you are using the beta version, you can do this from the ribbon:

At this point you will now have a fully functioning clone of the database created by the first developer. You can open the database from the GitHub folder and continue development on the database objects.

Committing Changes

1. Make some changes to the database on one of the development machines.
2. Click the button on the toolbar to Export Source Files.
3. You can review the changes at the source level using GitHub Desktop, or the VCS client of your choice.
4. When you are ready to commit the changes, simply commit (push) them to the repository.

The changes are now available to the other developers via the GitHub repository.

Loading Changes from Others

1. Export any pending changes to your current database before pulling down the changes from the repository. (You can leave the database open.)
2. Pull the new changes into your project at a source level using the VCS client.
3. Click Build From Source on the ribbon to rebuild the current database from source.

Since the database is already open at step 3, it knows where the source files are and builds the database without having to select the source folder each time. That's the basic workflow in a multi-developer scenario.

How you manage the flow of the changes is entirely up to you and what works best in your environment. Some people prefer having each developer work on their own branch, and use branch merging to bring changes across. Others take the simpler approach of just working on a single branch and coordinating with each other so both developers are not working on the same areas of the database at the same time.

The nice thing about managing all the changes and merging at the source level is that you can utilize conflict resolution tools like VS Code to work through issues where something was changed by both developers. After resolving the changes in the source, you can rebuild from source and have a build that matches whatever version you are working with at the source level.

Of course there are limits in how much overlap you can have between developers. Don't expect to have two developers making changes to the same form layout and expect to merge those changes together as source files. 😄

For non-technical users, I think the concept I would try to emphasize is that exporting source files is creating a set of instructions on how to build the database, based on the modifications they make through the user interface. It is important that they export their own changes before bringing in the changes from others. (This way they will see the conflicts in GitHub Desktop, or whatever client they are using.)

A few notes for you to keep in mind from a technical side:

• The index file should not be included in the git repository. You will see that it is excluded in the default .gitignore file. This is specific to that computer and developer and maintains the state between the database objects and the source files.
• I highly recommend separating the deployment from the development. In other words, don't work on the production database in a shared location and map the source files to a hard-coded repro path. That's just asking for trouble, in my opinion. 😄 It is much better to keep all the development all together (binary and source files) in your git repository, and deploy to end users as a separate step.
• If you don't already have another VCS client of preference, I recommend GitHub Desktop. It is being actively developed and quite user-friendly, even for users new to VCS.
• The add-in includes a conflict review/resolution screen to help users avoid overwriting modified source files or database objects, but this only going to show you the differences and give you the option to overwrite or skip. It's not going to help you make actual code changes on a line-by-line basis. I like to use WinMerge when comparing two files, so you might consider installing that (or something similar) on the development computers so they can compare changes if they hit a conflict.

Regarding some of your specific questions, let me touch on a couple things...

• Wherever possible, the add-in uses relative paths in relation to the database and source code locations. This is another reason it is helpful to just use the default export location so that the source is exported into a subfolder right under the database location. Bob and Alice can both use the same cloned repository on different paths because the source location is relative to the development database.
• As to the add-in requiring a database to be open, this was due to some instability if you attempted to load the add-in without any file open. I actually updated the ribbon today on the dev branch to work around this limitation. In the next beta release you will be able to activate the add-in even before opening any database.

I hope that helps! Again, hopefully I will be able to put together some tutorial videos in the future to give people a quick jump-start on the process when using the add-in for the first time.

Adam

[joyfullservice]

No problem! I should also note that you will need the beta (v4.x) version installed to build the dev branch. (You may have problems building version 4 if you are trying to build it from version 3.) Just install the latest beta version, then use that to build the dev branch. 👍

[joyfullservice]

I just want to clarify that this "chicken and egg" problem also exists for normal projects, and that there is at present not actually a way for me to follow the instructions you put under Client Setup (Other Developers) with an initial dev setup. There is no way to open Access and get access to the ribbon without opening a database, and there is no way to build the initial version of the database without access to the ribbon.

Sorry, I guess I never addressed that directly in any of my replies... 😄 The good news is that I have already resolved this in the dev branch, so any build going forward won't have that issue.

Technically, in order to load the add-in, a database file needed to be open. This could be any database, including a brand new blank database. (That is actually how I worked around it in the new version. I create a new blank database, load the add-in, then close and delete the temporary new database.) The way Access works is that one an add-in is loaded, it stays loaded in that instance of Access till the application is closed. This means that after the add-in is loaded, you could close the open database from the file menu, (without closing the whole application) and the ribbon buttons would still work.

When you click the Build From Source button, it first looks at the current database to see if it is connected to a set of source files. (Using a hidden database property that links to the source file path.) If it doesn't detect a link, it assumes this database is not connected to source files, and you must be wanting to build from a different set of source files. At that point it opens a folder picker where you can select the path to the source files for the database you want to build. The add-in will close the current (unrelated) database and focus on building a new database from the source files, in the path saved with the source files.

For example, let's say I have a project for MyDB.accdb. I could open a new database file on my desktop, just so I can load the add-in, then click Build From Source. Because the file on my desktop has never been built from source or exported as source files, there is no link to a folder of source files. Since the add-in as determined that I am not trying to build the file on my desktop, it opens the folder picker. I then browse to the MyDB.accdb.src folder, and select it. The add-in reads the source files, and determines that the new database is called MyDB.accdb and is located in the parent folder of the source files directory. (\..\MyDB.accdb). The add-in closes the desktop database without making any changes, and then goes to work building the MyDB database in the location specified by the source files. After the build completes, the MyDB.accdb file is open and ready for development.

With the changes I made yesterday, you won't have to do that intermediate step of using a blank database or other non version controlled database. You can just open Access and start the build right out of the gate. (Although you may notice that when the temporary database is closed, it will switch you back to the file menu, so you will want to click the Version Control tab on the ribbon to get back to the add-in screen.

Hope that helps!

[alandsidel]

Awesome, thanks!

For the time being, I just included a copy of the database in a new directory (and overrode the .gitignore for that directory) so anyone who clones can just copy the initial version into the parent directory and open it. I was a bit dismayed that the "build from source" seems to discard the data, but the merge option seems to keep it intact, so all good so far.

Once again thanks for your timely and informative replies!

[joyfullservice]

Regarding the table data, if you open your database, then go to the options screen, you can specify that the table data should be exported for the desired table(s). The next time you run an export, it should export the data as well as the structure for those tables.

Then, the next time you build that database, it should load in the exported data as a part of the build process. The options are saved in a per-project way, and should be included in your repository, so that change can be pushed out to the rest of the development computers.

On the technical side, those options are saved in the source files folder in vcs-options.json.

Also, just as an additional note, the Merge Build functionality is a brand new feature that has been in the works for a long time. It is especially helpful for very large complex databases where a full build can take several minutes to run, but a merge might only take a few seconds. A full build is the most robust and tested approach we have been running for years, but merge builds is something I literally just got functional in the last couple weeks. If it works for what you are needing, go for it! Otherwise you can always fall back to the more battle-tested full builds if you run into problems with the merge. (Feel free to create issues if you have problems with merges or anything else.)

[alandsidel]

Thanks. So far the merge has worked ok. I don't expect this to work when importing schema changes, it's a bit much to ask a tool like this to invent data migrations, but if it can rebuild queries and forms without affecting the data, that would be great. I imagine once the people who are going to use this actually have it in their hands, it won't be long before they all have separate test/dev data in their local copies.

I'm not much of an Access guy but I imagine there's a built in way to export/import data as csv, xml, or something like that separate from this plugin. I'll experiment with that and add those files to the .gitignore if I find something I think the end users will be able to live with.