"delete tbl" vs clear_table()

[vpax]

We have a BiF, clear_table(), that clears the contents of a table in situ. Normally you can do this just by assigning an empty table() constructor to the table you want to clear, but the BiF is solving a problem where the table is available only by shallow-copy, as an argument. The one use in the standard scripts is here:

zeek/scripts/base/protocols/dns/main.zeek

Line 200 in cc646f6

clear_table(msgs);

I'd like to deprecate the BiF and replace it with expanded semantics for the delete statement: if you give it an entire table or set, e.g. delete tbl, it clears it (keeping the attributes). This matches functionality that awk provides (FWIW, it was the inspiration behind Zeek having a delete statement).

Why do I care? Because I'm working on a fix/improvement for script optimization, for which it's very helpful to know that none of our BiFs alter script-level non-opaque state. AFAICT, this is the only BiF that does so.

[awelzel]

AFAICT, this is the only BiF that does so.

sort() and resize() might be two other candidates? Option::set() setting arbitrary option identifier's value, too? External plugins are free to poke at script state as they want - one can always provide their own clear_table() like bif and do much worse things. Clearing or populating script state from a plugin may be a bit weird, but not completely crazy (a stats table or vector that's kept up-to-date behind the scenes).

[vpax]

Thanks for flagging those others. My plan is to implement IsPure() for BiFs, with a default of false, and then the optimizer will use that to tell which BiFs have no script-visible side effects. (I believe the semantics of IsPure() have never been well-defined nor implemented for ASTs, alas, and really we want multiple notions, such as idempotency vs. no-script-visible-side-effects. But this will be a start.)

Even given that, I'd still rather that delete tbl works like it does in awk, and get rid of the (IMHO) clunky clear_table().

[vpax]

Well, rats - I looked into the IsPure() approach and don't see a non-clunky, non-error-prone way to do it. So I will (sigh) instead use an explicit list of safe BiFs.

But I'd still like to get rid of clear_table()!

[0xxon]

Note that, to my understanding, clear_table fulfills a specific use-case which cannot be substituted by assigning a new table();

Specifically when a table has attributes, I think these are bound to the instance and will be wiped by assigning an empty table. To give an example, when running the following script...

module TEST;

function efu(reqs: table[string] of count, s: string): interval
	{
	print s;
	return 0secs;
	}

export {
       global t: table[string] of count &create_expire=0sec &expire_func=efu;
}


event zeek_init()
	{
	t = table();
	t["hi"] = 5;
	}

no output will occur on stdout. When removing the table assignment, the expire_func is called.

[vpax]

Yep and I noted that above (that delete tbl should preserve attributes). Basically, all I'm asking for is syntax to do this, I think it's clunky for it to be a BiF, and a tad strange that you call clear_table() to clear sets, too. (If it's a statement then it could also clear vectors - not sure how often that need arises, but There You Go.)

[0xxon]

You are correct, reading is apparently hard. Sounds good to me.

[awelzel]

Delete to remove all elements of a vector, table or set will be in Zeek 7.0.

[vpax]

I definitely like this ... That Said, it made me wonder about the semantics of:

global tbl: table[count] of table[bool] of string;
tbl = table([3] = table([F] = "false"), [5] = table([T] = "true"));
print tbl;
delete tbl[3];
print tbl;

So does delete tbl[3] clear the table element tbl[3], or remove that element from tbl entirely. Looks like currently it does that latter. I sort of think "you make it complex like that, yeah it's confusing", and don't really have a problem with the semantics being remove-entirely-don't-just-clear ... but thought I'd tee it up to see if I'm missing a more unifying perspective.

I.e. is the clear-table semantics limited to delete var for an atomic var? That's the current semantics, as best as I can tell, and I'm fine with that - but wanted to confirm before I fix this for this month's script optimization.

[awelzel]

That's the current semantics, as best as I can tell, and I'm fine with that - but wanted to confirm before I fix this for this month's script optimization.

I did not think about this :-/ Delete with an IndexExpr will remove the element, delete on a NameExpr will clear it.. that's the implementation. But with your example and thinking about the work-around, realize this is somewhat subtle/quirky/asymmetric. ``` # This can be used to clear the sub-table, while delete tbl[3] removes it. local e = tbl[3]; delete e; # ...this continues to work clear_table(tbl[3]); ``` I don't think we can/should change the IndexExpr delete to be yield-type dependent. Given past behavior, that would be more surprising. Only thought is maybe special syntax like `delete tbl[]` and then `delete[3][]`? I see that awk implements multi-dimensional array/table via string-concat of indices, so can't use something from there as a model :-/

[vpax]

Cool, I'll implement it based on AST node type. (Hmmm and there's also delete foo$bar where bar is an optional field of type table :-P ). Given there are ways to still achieve what's quite likely a rare case, this is fine by me.