[Enhancement] The current status shows the JSON code

[nanochess]

Is there an existing issue for this?

• I have searched for existing issues and did not find anything like this

Describe the bug

The current status page shows all of the JSON cases that should be preprocessed before display.

What you expect

I expect a shorter webpage and not all the cases description.

Environment

• OS: macOS 12.7.6
• Device: Macbook Air M1
• Compiler: Not used.

Anything else?

No response

[xexyl]

This is what status.json is for. Which it links to towards the top.

The status.json is about the statuses not the actual status (except at the top).

[xexyl]

Maybe it could be higher up? But what it might say is beyond me at this time.

[xexyl]

BTW: the top also shows the status.json file.

Maybe there is a misunderstanding?

[xexyl]

What do you wish to be done @lcn2 ?

I can take care of it if you let me know. Thanks.

[lcn2]

Perhaps the Format section down to the horizontal line could be moved to the FAQ?

Then instead of all that Format section text, one could add a single:

See also the
FAQ on "..."
for more information about [status.json](status.json) file format as well as information about the **contest_status**.

Please consider something like that as a potential solution to the enhancement suggestion by @nanochess, @xexyl.

[xexyl]

Okay but which FAQ? If we add an FAQ with all the details of the status.html that's going to be a long FAQ.

I would guess I'm not following you right? Please clarify.

[lcn2]

Okay but which FAQ? If we add an FAQ with all the details of the status.html that's going to be a long FAQ.

I would guess I'm not following you right? Please clarify.

How about either adding it as a new FAQ 10.11 that describes the contest status (i.e., pending, open, judging, closed)
and a new FAQ 10.12 that describes the status..json file format?

Or better still add, a new FAQ Section 12 about "IOCCC status", with a new FAQ 12.0 that describes the contest status (i.e., pending, open, judging, closed)
and a new FAQ 12.1 that describes the status..json file format?

A new section at the bottom of the FAQ might be a better place for this information.
Of it not at the bottom, move the "History of the IOCCC" down to FAQ 12 and make the new "IOCCC status" section 11.

[xexyl]

Okay but which FAQ? If we add an FAQ with all the details of the status.html that's going to be a long FAQ.
I would guess I'm not following you right? Please clarify.

How about either adding it as a new FAQ 10.11 that describes the contest status (i.e., pending, open, judging, closed) and a new FAQ 10.12 that describes the status..json file format?

This is far more complicated than adding an FAQ and updating a markdown file though. See below.

Or better still add, a new FAQ Section 12 about "IOCCC status", with a new FAQ 12.0 that describes the contest status (i.e., pending, open, judging, closed) and a new FAQ 12.1 that describes the status..json file format?

A new section at the bottom of the FAQ might be a better place for this information. Of it not at the bottom, move the "History of the IOCCC" down to FAQ 12 and make the new "IOCCC status" section 11.

Well that would be better .. assuming that we don't have any links using the number system (which we shouldn't). Personally I don't see the issue here but since it was brought up it's obviously an issue to some.

The question is what should remain in status.html too. I guess the top part up through the contents of status.json.

A much worse problem is this is not a simple matter of updating status.md! Significant changes to bin/gen-status.sh will have to be done too. Right now I'm hoping to look at the other repo to get some of the things you needed done (and those things are more important as they are required for the next contest).

[lcn2]

This is actually an enhancement that can be improved on either pre or post IOCCC28 opening. The idea raised by @nanochess is valid. The solution, however, should not delay the opening of IOCCC28.

[xexyl]

This is actually an enhancement that can be improved on either pre or post IOCCC28 opening. The idea raised by @nanochess is valid. The solution, however, should not delay the opening of IOCCC28.

I thought those things too. Glad you agree. Just put in the other repo some thoughts/questions about an enhancement. I have to do other things now but if I don't do more today I'll certainly tomorrow morning.

[xexyl]

As for when the contest is in open status I won't very likely be working here. It is not clear even what I'll be doing. But I can certainly look into it after IOCCC28 though I do have some plans also. Plus I (and hopefully you can help some) have the new tools (after discussion) in jparse and it would be nice to fix the UTF-8 issue that @SirWumpus reported (and did some great work towards!).

Anyway maybe I can get to this in the months following IOCCC28.

[SirWumpus]

it would be nice to fix the UTF-8 issue that @SirWumpus reported (and did some great work towards!).

While that code started out as a bit of plod, it turned out to be pretty interesting study. Hope you use it at some point.

[lcn2]

it would be nice to fix the UTF-8 issue that @SirWumpus reported (and did some great work towards!).

While that code started out as a bit of plod, it turned out to be pretty interesting study. Hope you use it at some point.

I would highly encourage you, @xexyl, to incorporate that code into the jparse repo, after IOCCC28 opens. Post IOCCC28, we can sync such a change into "the other repo".

We still wonder why, assuming that your method works @SirWumpus (and we have found no case where it doesn't), why this isn't the preferred method for processing UTF-8.

Thanks again, @SirWumpus

On NetBSD wc

Your approach, @SirWumpus, to managing non-UTF-8 byte sequences should be the norm for tools such as wc(1), grep(1), etc.

The NetBSD folks appears to have broken fundamental tools such as wc(1) by trying to pedantically "whining about unexpected byte sequence on stderr". From what I'm told behind the scenes by one of the original UC Berkeley BSD crew, some UTF-8 zealot(s) wanted to put such "whining on stderr" deep into libc as well. There were objections such a change, of course. But certainly someone was successful in "buggering" (to use the cant Australian expression)wc(1), sadly.

We wonder if the folk who damaged wc(1) did the same to strings(1) or grep(1) or ... We hope not.

See bin/58014: wc no longer works with binary files. Who knows if they will fix the "buggering" of tools like wc(1) or not. Documentingthe misfeature in a man page doesn't fix the problem, in our opinion.

BTW: Unless or until it is fixed, we will consider NetBSD wc(1) broken and would politely close as "won't fix", any bug reports that arise from their broken tool. ... but that's just our opinion too. 🤓

An approach to tools such as wc(1) we would have recommend is to add a non-standard flag such as, say --UTF8_valiation, which would turn on reporting of "invalid byte sequences". This by default, no such "whining about unexpected byte sequence on stderr" would occur unless someone explicitly asked for it. And in the man page, warn that --UTF8_valiation is for UTF-8 testing purposes only, and that it is NOT widely supported, and use of --UTF8_valiation, could case failures on other UNIX systems.

UPDATE 0

The compiler warning about strncpy(3) appear to be another example of someone trying to enforce what they thing is the way to code, via whining in compiler warnings.

While not exclusive to BetBSD, there are folk who consider it a mission and a "good feature" to make code that uses strncpy(3) harder to compile.

Our approach in the calc repo was to introduce modify code that used potentially unsafe functions and then to effectually ban future use in the calc code via banned.h.

This is a case where an application makes a choice to move away from the use of a potentially problematic function.

If a compiler is going to do that, they should add a --whine_about_unsafe_functions to do that instead of forcing the compiler warnings as a default.

UPDATE 1

Then there are the folk who want to stop people using fgrep(1) and egrep(1) by adding whining messages to stderr.

sigh

[xexyl]

it would be nice to fix the UTF-8 issue that @SirWumpus reported (and did some great work towards!).

While that code started out as a bit of plod, it turned out to be pretty interesting study. Hope you use it at some point.

I would highly encourage you, @xexyl, to incorporate that code into the jparse repo, after IOCCC28 opens. Post IOCCC28, we can sync such a change into "the other repo".

Okay but you used 'I'. There's a problem there! I won't go to the usual place in 'The Black Gate Is Closed' but I'm tempted to!

We still wonder why, assuming that your method works @SirWumpus (and we have found no case where it doesn't), why this isn't the preferred method for processing UTF-8.

The reason is quite simply that I haven't had the time to try and integrate it into the library. I believe it might take a fair amount of work and I'm trying to work on the contest code too. I will certainly be looking at it however.

Of course you're more than welcome to help with this task as well *hint* :-) But we both have more important things to work on right now. As for after IOCCC28 like I said I do have some plans but I hope to start looking at jparse more. Of course during the contest my birthday will be here (as you know) so that day (a Friday this year) and the next day I likely won't be doing much (actually almost certain of it).

Thanks again, @SirWumpus

My thanks as well!

On NetBSD wc

...

Very interesting commentary that normally I might reply to but trying to work on txzchk. I would be inclined to agree with your points however.

[lcn2]

Okay but you used 'I'. There's a problem there! I won't go to the usual place in 'The Black Gate Is Closed' but I'm tempted to!

We use I explicitly because we were not speaking for the IOCCC.

The IOCCC judges won't look down on a submission that used strncpy(3), or that feeds binary stuff thru the wc(1), for example.

[xexyl]

Okay but you used 'I'. There's a problem there! I won't go to the usual place in 'The Black Gate Is Closed' but I'm tempted to!

We use I explicitly because we were not speaking for the IOCCC.

Oh! I always assumed that you did it because a lot of people do (I sometimes do too as does ironically and amusingly Gollum, as you know).

But of course I was just going to the old joke between us anyway.

The IOCCC judges won't look down on a submission that used strncpy(3), or that feeds binary stuff thru the wc(1), for example.

Of course. And neither would you necessarily look down on an implementation of wc(1), or at least you wouldn't have. I guess you would now since Dave Burton won it in 2019 (best one-liner iirc).

[xexyl]

On NetBSD wc

Your approach, @SirWumpus, to managing non-UTF-8 byte sequences should be the norm for tools such as wc(1), grep(1), etc.

No surprises there and a great compliment to him too (and well deserved)! I still have a lot of work to do with integrating it of course but that can come down the road.

The NetBSD folks appears to have broken fundamental tools such as wc(1) by trying to pedantically "whining about unexpected byte sequence on stderr". From what I'm told behind the scenes by one of the original UC Berkeley BSD crew, some UTF-8 zealot(s) wanted to put such "whining on stderr" deep into libc as well. There were objections such a change, of course. But certainly someone was successful in "buggering" (to use the cant Australian expression)wc(1), sadly.

I am reminded of the linux man page for accept(2) talking about POSIX changing the int to the wrong type (which is broken) and then to save face they added the socklen_t. Linus wrote about that. Linus stated that some people complained (as they should) but not enough people.

Cant: a funny word that 'auto-"correct"' likes to change to 'can't'. Just like 'wont' (sometimes at least).

But as you know the best one is when one changed erroneously 'shellcheck' to ... 'spellcheck'!

We wonder if the folk who damaged wc(1) did the same to strings(1) or grep(1) or ... We hope not.

Hopefully not.

See bin/58014: wc no longer works with binary files. Who knows if they will fix the "buggering" of tools like wc(1) or not. Documentingthe misfeature in a man page doesn't fix the problem, in our opinion.

In many cases documenting it is just a way to say we know of it but we don't care indeed. Of course in other cases it's perfectly fine.

BTW: Unless or until it is fixed, we will consider NetBSD wc(1) broken and would politely close as "won't fix", any bug reports that arise from their broken tool. ... but that's just our opinion too. 🤓

An approach to tools such as wc(1) we would have recommend is to add a non-standard flag such as, say --UTF8_valiation, which would turn on reporting of "invalid byte sequences". This by default, no such "whining about unexpected byte sequence on stderr" would occur unless someone explicitly asked for it. And in the man page, warn that --UTF8_valiation is for UTF-8 testing purposes only, and that it is NOT widely supported, and use of --UTF8_valiation, could case failures on other UNIX systems.

That would be nice wouldn't it? But alas we do not live in an ideal world. But maybe that's a good thing. If we did live in an ideal world would we really be happy? Certainly many people would not be but we also might not bother doing anything at all. That is my belief based on what I have seen in this world.

UPDATE 0

The compiler warning about strncpy(3) appear to be another example of someone trying to enforce what they thing is the way to code, via whining in compiler warnings.

While not exclusive to BetBSD, there are folk who consider it a mission and a "good feature" to make code that uses strncpy(3) harder to compile.

As for strncpy(3) versus strcpy(3) I recall in some versions of the linux man pages they talk about the problems with the former function. It should be obvious: it can lead to non-terminated strings which is also dangerous. Plus if you ensure that you have enough space you shouldn't need it. And if you know the maximum length for strncpy(3) you should be able to allocate enough space anyway! Certainly in some cases it can be valuable though but that should be up to the programmer. I deem that warning also dubious. A similar warning if not that exact warning was a huge pain to me in another project. I just ended up disabling the warning if memory serves right.

Our approach in the calc repo was to introduce modify code that used potentially unsafe functions and then to effectually ban future use in the calc code via banned.h.

This is a case where an application makes a choice to move away from the use of a potentially problematic function.

If a compiler is going to do that, they should add a --whine_about_unsafe_functions to do that instead of forcing the compiler warnings as a default.

I'd like that. The warning about use of pointers in some contexts is beyond ludicrous and probably the worst one I have seen. To warn about pointers in C (used as/related to an array) is insane!

And the supposed misleading formatting one is also ludicrous. I had to 'fix' code just to shut that warning up years ago as at the time there seemed to be no way to disable it. Ironically by introducing such things it's more likely to cause problems because you have to fix perfectly valid code that might have got a messed up formatting for some reason. The problem is when you warn about dubious things people might actually disable all warnings or ignore them and then they will do something that actually is bad.

UPDATE 1

Then there are the folk who want to stop people using fgrep(1) and egrep(1) by adding whining messages to stderr.

sigh

That bothered me at first but only because it was used in a cronjob that runs very frequently. I just changed it. It's kind of annoying but not as bad as say systemd (is anything that bad?).

[xexyl]

it would be nice to fix the UTF-8 issue that @SirWumpus reported (and did some great work towards!).

While that code started out as a bit of plod, it turned out to be pretty interesting study. Hope you use it at some point.

Glad it was good for you too! Thanks. I hope I can use it too. I guess it'll involve some study as well and also some work to try and integrate it into the parser. That'll be fun!

[SirWumpus]

We use I explicitly because we were not speaking for the IOCCC.

Figured you discarded the (royal) we when you weren't talking for the Judge Collective (or when you took your crown off when going to the throne loo) .

[xexyl]

We use I explicitly because we were not speaking for the IOCCC.

Figured you discarded the (royal) we when you weren't talking for the Judge Collective (or when you took your crown off when going to the throne loo) .

Ah yes. When it comes to humour the bog always works well for those who don't mind such sort of humour.

[SirWumpus]

Your approach, @SirWumpus, to managing non-UTF-8 byte sequences should be the norm for tools...

Purists probably assume binary will use more efficient Base64, rather than encode arbitrary binary as though it is encoded UTF16. I think I noticed something about UTF8 errors in the C standard wide-character functions mblen, mbtowc that they fail with an error for invalid encodings, which is fine in some places. Purists probably assume you should only use the C standard functions and not roll you're own, which leads the mess with wc.

[xexyl]

Your approach, @SirWumpus, to managing non-UTF-8 byte sequences should be the norm for tools...

Purists probably assume binary will use more efficient Base64, rather than encode arbitrary binary as though it is encoded UTF16. I think I noticed something about UTF8 errors in the C standard wide-character functions mblen, mbtowc that they fail with an error for invalid encodings, which is fine in some places. Purists probably assume you should only use the C standard functions and not roll you're own, which leads the mess with wc.

Okay but aren't all wcs a bit messy?

[SirWumpus]

When it comes to humour the bog always works well

Unless you are an Irish peat moss cutter ("bog men") you probably take some offence.

[xexyl]

When it comes to humour the bog always works well

Unless you are an Irish peat moss cutter ("bog men") you probably take some offence.

Depends on if one has a sense of humour I'd say. You could even add to this!

[SirWumpus]

We still wonder why, assuming that your method works @SirWumpus (and we have found no case where it doesn't), why this isn't the preferred method for processing UTF-8.

Thanks again, @SirWumpus

@lcn2 Glad you like it. I couldn't understand why @xexyl kept saying its really complex. I had to see for myself what the fuss was about and wrote utf8str et al. Short, sweet, to the point. (Maybe I should submit it to IOCCC28 just for fun - But its not that fancy it probably be passed over.)

[xexyl]

We still wonder why, assuming that your method works @SirWumpus (and we have found no case where it doesn't), why this isn't the preferred method for processing UTF-8.

Thanks again, @SirWumpus

@lcn2 Glad you like it. I couldn't understand why @xexyl kept saying its really complex. I had to see for myself what the fuss was about and wrote utf8str et al. Short, sweet, to the point. (Maybe I should submit it to IOCCC28 just for fun.)

You misunderstand. The lower level functions in the parser are complex so I have to work it into that. Your code seems simple enough yes.

[SirWumpus]

You misunderstand. The lower level functions in the parser are complex so I have to work it into that. Your code seems simple enough yes.

You and I should talk more about that after IOCCC28. If a parser is that complex, then its sounds a little over engineered. I've avoid looking at that code for now cause I might get "re-writers itch".

[xexyl]

You misunderstand. The lower level functions in the parser are complex so I have to work it into that. Your code seems simple enough yes.

You and I should talk more about that after IOCCC28. If a parser is that complex, then its sounds a little over engineered. I've avoid looking at that code for now cause I might get "re-writers itch".

Sure .. though that function core I did not write myself. Do feel free to take a look if you wish. Since you wrote that code you know it better. Actually it's a bit of a mix. To fix one bug it involved a couple other UTF-8 functions including one from (I think) the linux kernel (but modified). It is the parsing of the string and calculating the necessary size that can be complex. Part of that is the fault of the JSON requirements.

But we can certainly discuss it if you are up to it. Would be great.

[xexyl]

You misunderstand. The lower level functions in the parser are complex so I have to work it into that. Your code seems simple enough yes.

You and I should talk more about that after IOCCC28. If a parser is that complex, then its sounds a little over engineered. I've avoid looking at that code for now cause I might get "re-writers itch".

Sure .. though that function core I did not write myself. Do feel free to take a look if you wish. Since you wrote that code you know it better. Actually it's a bit of a mix. To fix one bug it involved a couple other UTF-8 functions including one from (I think) the linux kernel (but modified). It is the parsing of the string and calculating the necessary size that can be complex. Part of that is the fault of the JSON requirements.

But we can certainly discuss it if you are up to it. Would be great.

.. of course it could also simply be because I haven't had the energy and time and patience to look into it more carefully but anyway hopefully we will get it sorted out once IOCCC28 is over.

Anyway you are certainly welcome to look at it if you like and I would be happy to look at any pull request. The same make rules that are in mkiocccentry prep and release do the same things btw.

Hope you have a great night @SirWumpus and thanks again!

[xexyl]

BTW @SirWumpus: I just looked at it a bit but it seems macOS does not have uchar.h! I'll mention you over at that repo with some updates so as to not post OT stuff here.

[SirWumpus]

BTW @SirWumpus: I just looked at it a bit but it seems macOS does not have uchar.h! I'll mention you over at that repo with some updates so as to not post OT stuff here.

uchar.h has been in the standard since C11, so MacOS is very behind.

[lcn2]

BTW @SirWumpus: I just looked at it a bit but it seems macOS does not have uchar.h! I'll mention you over at that repo with some updates so as to not post OT stuff here.

uchar.h has been in the standard since C11, so MacOS is very behind.

macOS does have uchar.h. It is: <unicode/uchar.h>

[SirWumpus]

uchar.h has been in the standard since C11, so MacOS is very behind.

macOS does have uchar.h. It is: <unicode/uchar.h>

But the standard makes no mention of needing -I/usr/include/unicode or any subdirectory to hold standard headers. It might exist, but seems like to use it you need to muck about with #ifdef or feature macros.

[xexyl]

BTW @SirWumpus: I just looked at it a bit but it seems macOS does not have uchar.h! I'll mention you over at that repo with some updates so as to not post OT stuff here.

uchar.h has been in the standard since C11, so MacOS is very behind.

macOS does have uchar.h. It is: <unicode/uchar.h>

That curious because I used locate uchar.h.

Oh well. Thanks.

[lcn2]

We believe we have addressed all of the current questions that still need answering at this time. If we've missed something or something else needs to be clarified, please ask again.

[lcn2]

QUESTION

Are we in a position to close this issue today in order to meet the soft code freeze deadline?

NOTE after today, we will about 5 more days to:

• polish the rules, guidelines and FAQ
• test things
• fix only critical code bugs
• reg the release numbers
• announce a hard code freeze
• open IOCCC28

So let us REALLY try to be in a position to close this issue soon, if possible.

UPDATE 0

We do understand that this issue addresses "rules and guidelines", so perhaps this issue can be closed and let the talks of "polishing the rules and guidelines" until IOCCC28 opens?

[lcn2]

Commit fadb1dd resolved this issue by moving information about the fields of status.json to FAQ 10.11.