Metadata less

[ThierryGoubier]

Dale,

here is the metadata-less version of gitfiletree.

To summarize the features:

• It's not reading the monticello.meta/version file anymore,
• package version info is extracted from the git commit ID choosen (author, time stamp, message),
• package ancestry is rebuilt from the package-related commits and the full repository log history,
• package version numbers are extracted from the list of package-related commits,
• package id is randomly created everytime the repository filenames are loaded,
• all tests are green on my system and pharo2.0,
• no changes to the package writing code, it is the filetree code.

It does change the way the repository contents look like (particularly history, version numbers and author), so if you can validate the way it looks (or if others may have a look as well), I'd be happy to have your return.

[camillobruni]

this looks very nice, I just had a look at the repo itself and it makes working with filetree and normal git tools much more fluent. BTW, this was the reason I never used filetree for the pharo-core export since there was simply too much noise to recognize any crucial information.

nice work

[ThierryGoubier]

I'm now thinking that, with this code, I could remove the methodProperties.json file as well, since the Author and timestamp are recorded by the git commit.

Thierry

[dalehenrich]

I haven't had a chance to review your code yet, but the methodProp erties.json file is only there to preserve Monticello meta data ...

Dale

----- Original Message -----

| From: "Thierry Goubier" [email protected]
| To: "dalehenrich/filetree" [email protected]
| Sent: Monday, July 22, 2013 8:48:16 AM
| Subject: Re: [filetree] Metadata less (#93)

| I'm now thinking that, with this code, I could remove the
| methodProperties.json file as well, since the Author and timestamp
| are recorded by the git commit.
| Thierry
| —
| Reply to this email directly or view it on GitHub .

[ThierryGoubier]

Oh. I believed it was used to record the timestamp and creator as recorded in the changes file ? Because there is also the properties.json file as well... But I don't think I should try to get rid of this one.

[dalehenrich]

The methodProperties.json file records meta data that is part of the Monticello definitions ... Git records the commiter and commit data for each method (file) change, so it should be possible to reconstruct the method meta data from Git and elminate the methodProperties.json file.

The properties.json files manage FileTree information and should not/cannot be eliminated...

Dale
----- Original Message -----

| From: "Thierry Goubier" [email protected]
| To: "dalehenrich/filetree" [email protected]
| Cc: "Dale Henrichs" [email protected]
| Sent: Monday, July 22, 2013 10:08:01 AM
| Subject: Re: [filetree] Metadata less (#93)

| Oh. I believed it was used to record the timestamp and creator as
| recorded in the changes file ? Because there is also the
| properties.json file as well... But I don't think I should try to
| get rid of this one.
| —
| Reply to this email directly or view it on GitHub .

[ThierryGoubier]

Ok, the next two commits are without the method properties metadata. Now, the timestamp is exclusively the data stored by git upon the commit (i.e. not the real time the method is written, but when it's committed to the git repository).

The code is a bit rough, in that it still reads the methodProperties metadata if it is present, but it doesn't do anything with it. If you think it is Ok, I'll clean the code.

[dalehenrich]

Yeah, we need the timestamp from the definition itself, since it appears to failing the first assertion for timestamp tests ... Ugly, but necessary when it only fails on travis:(

[ThierryGoubier]

Well, no chances it can work. I don't have the same issue33 repo as on Travis.

On travis, the git sha1 is 4058b81, and I have only one version in the history (Author: DaleHenrichs Time: 28 June 2012, 3:07:37 pm)

On my branch, as merged from the pharo 2.0 branch, I have:
two versions in the history, the git sha1 are b36f0a2 and f8daa1b, DaleHenrich 16 June 2012 for both.

I think that if I create an issue33.2 repo and update the timestamps, then it should work through the merge. Now, let's look at the next test case.

Or does that point to a problem in the way the test repos are copied to travis ?

[ThierryGoubier]

Same here.

• ver01 should be 65e9904, on travis it is 4058b81.
• ver02 should be 8d225c9, on travis it is 4058b81.

I can add code in the tests to force an alternative checkup on the commitId available in Travis, but it doesn't make sense, isn't it?

[dalehenrich]

Okay ... when travis does a run it does a merge of your branch and the pharo3.0 branch and does a commit .... I think the lesson here is that depending upon a SHA is not quite the same as a UUID ... if you were to checkout a specific commit of the test directory, you might have more luck, but presumably a change to a readme will cause the directory's SHA to be changed?

Dale
----- Original Message -----

| From: "Thierry Goubier" [email protected]
| To: "dalehenrich/filetree" [email protected]
| Cc: "Dale Henrichs" [email protected]
| Sent: Wednesday, July 24, 2013 1:55:58 PM
| Subject: Re: [filetree] Metadata less (#93)

| Same here.

| * ver01 should be 65e9904 , on travis it is 4058b81 .
| * ver02 should be 8d225c9 , on travis it is 4058b81 .

| I can add code in the tests to force an alternative checkup on the
| commitId available in Travis, but it doesn't make sense, isn't it?
| —
| Reply to this email directly or view it on GitHub .

[ThierryGoubier]

Ok, if it merges against pharo3.0 instead of 2.0 I may be in trouble.

CommitID is specific to the package, not to anything in the surroundings. If the Readme is changed, its counted as a new version of the package.

I've added the nonsense git sha1 from Travis as special timestamps and commits to try to get the tests to go green. I'm done for tonight.

[dalehenrich]

Perhaps a different testing strategy is called for? Presumably you don't really care about the UUID of the packages and basing the UUID on the SHA of the commit should be enough. The UUIDs should be equal if a package is loaded from two different repositories checking out the same commit ... so that is what is important ... so perhaps we shouldn't care about the UUID so much (other than to validate that two different checkouts of the same commit give the same UUID ... for the method time stamps the (unfortunate) right answer is to use the timestamp of the last commit of that package .... then at least you will have stable timestamps ... which I think are important and more correct ...

Fresh start in morning:)

[ThierryGoubier]

Dale, I don't test against the package UUID, just against the git data (commitID, author and date). All that you describe as should be done is the way gitfiletree is doing it (package uuid is sha1 hash of git commit id + package name, timestamp is git commit timestamp, author is git commit author).

I strongly believe travis makes a mess of the merge with the test data, and that I am unable to get anything reasonable out of my git queries.

[ThierryGoubier]

I know the culprit: Travis L4, git clone -depth=50. This cuts short the git history and removes the commits I was checking for: f8daa1b, 65e9904, 8d225c9.

I leave it to you to change the travis CI script and revert this request to 523c789, or keep as it is now since it has additional tests to detect a future git copy mistake, all tracing code has been removed in 62c14ae (but I would suggest changing the ci script, otherwise those tests will start failing in a year or so...).

[dalehenrich]

Added builderCI issue for the clone depth bit ... I think parameterizing the depth makes sense since very few tests need to copy the whole repo

[ThierryGoubier]

Thanks! It's a relief to have been able to solve it.

[dalehenrich]

Yeah, that was a great bit of sleuthing!