Set default of flood_advert_interval to 48h, minimum interval to 12h

[mattzzw]

This should help reducing congestion/noise in big meshes.
Also applies to room server.

[Fidonet127]

This is a bad idea as a default. New users won’t see anything for potentially 48 hours in sparse areas, leading them to think there is no MC there. If anything put it out in your nets to change or have some automatic determination.

[jbrazio]

I'm not too comfortable with this PR.
Not all meshes are big, 12h is a good sensible all around default value.

I would argue that:

1. it should be the default value,
2. but also the minimum value for a flood advert.

[mattzzw]

12h is a good sensible all around default value.

How do you come to this conclusion?

Let's look at the facts:

• Repeater flood adverts serve absolutely no technical purpose. They are propagated noise that very quickly adds up in a mesh network designed for data minimalism and robustness. This is an important lesson we learned from MT.
• There is a function to discover nearby repeaters with a zero hop packet. Use that if you want to see repeaters around you.
• Even in medium size meshes the data amount (and airtime consumed) by repeater adverts already makes up roughly 50% of the traffic. (see statistic below)
• Small meshes will absolutely not benefit from a higher flood advert rate.

[jbrazio]

Small meshes will absolutely not benefit from a higher flood advert rate

People like to see repeaters fast when they join a small mesh. It is a "social thing" that makes meshes grow. Remember that traceability is a core feature of the core and monkey brains no likey "unknonwn hops".

Your graph is flawed, you are just graphing "control traffic" which needs to be put into perspective with the main traffic this mesh should transport, private and group messages. If then advert ratio still stacks to 50% of your total traffic you have other problems at hand.

[mattzzw]

People like to see repeaters fast when they join a small mesh. It is a "social thing" that makes meshes grow. Remember that traceability is a core feature of the core and monkey brains no likey "unknonwn hops".

I get that point. I propose to change the user experience on the app side, not with useless noise on the network side. E.g. put a big red "discover nearby repeaters"-button in the app, a little intro tutorial for newbees. Or a message to be patient.

Your graph is flawed, you are just graphing "control traffic" which needs to be put into perspective with the main traffic this mesh should transport, private and group messages.

My bad, I was not careful enough while screenshotting this from https://analyzer.letsme.sh/stats

If then advert ratio still stacks to 50% of your total traffic you have other problems at hand.

Exactly. You can go to letsme.sh/stats and see for yourself.

[ViezeVingertjes]

If then advert ratio still stacks to 50% of your total traffic you have other problems at hand.

It does so easily as Advert packets are much larger in size than messages, thus taking up way more airtime etc. Currently we've seen total halts of the network due to just those flood adverts, so its something that needs to be addressed soon. This is just a temporary fix to reduce it a little bit, but its a start.

Also opened a discussion topic here -> #1223

[liamcottle]

I would personally prefer to keep the minimum allowed flood advert interval as 3 hours.

Setting it to 12 hours minimum now means if you miss 2 adverts, (likely when a repeater is a fair few hops away), you might have to wait several days to know if your repeaters are still alive.

The firmware currently defaults to 12 hours, but having the ability to set it lower in smaller self-controlled meshes is very useful.

I understand that in larger meshes the lower intervals can be problematic, but it's not set to 3 hours by default, so if a user changes it to that, there must be a reason.

It's also noted there's been discussion where 12 hours is even too frequent, in extremely large meshes. So some other solution should be introduced to resolve the issue of too many adverts, rather than hard limiting the minimum.

Just a note on the PR, if you're wanting to increase the minimum advert intervals, you'd need to change the validation in src/helpers/CommonCLI.cpp, rather than inside the updateFloodAdvertTimer function.

MeshCore/src/helpers/CommonCLI.cpp

Line 374 in 2228214

if ((hours > 0 && hours < 3) || (hours > 48)) {

The adverts are indeed large packets, but we want to prevent spoofing on the network, so the signatures need to be present. If the adverts are limited in minimum duration, there would need to be some other way to check that repeaters are alive, but this would also introduce more traffic. My opinion is that the adverts might as well serve this purpose.

In larger meshes, the advert frequency could probably be automatically adjusted so it's not so frequent, or the adverts could be reduced in priority. I believe I saw some discussion about that idea already.

[ViezeVingertjes]

but it's not set to 3 hours by default, so if a user changes it to that, there must be a reason.

That's true, but they see it in the settings and then be like; let's max everything out, not knowing what impact it has. So if we give the option, they surely will abuse it.

But indeed, i think there could be more ways to approach this, like hard enforcing a certain amount of adverts per node per x time window etc on the repeater side. As people with companions also like to spam flood adverts a lot and we also see repeaters in bootloops ddossing the mesh etc.

[jbrazio]

I'm not sure everyone is aware that the core has already a packet prioritization algorithm, where type advert is bottom queue.

void Mesh::sendFlood(Packet* packet, uint32_t delay_millis) {
  // ... setup code ...
  
  uint8_t pri;
  if (packet->getPayloadType() == PAYLOAD_TYPE_PATH) {
    pri = 2;
  } else if (packet->getPayloadType() == PAYLOAD_TYPE_ADVERT) {
    pri = 3;   // de-prioritie these
  } else {
    pri = 1;
  }
  sendPacket(packet, pri, delay_millis);
}

[ViezeVingertjes]

I'm not sure everyone is aware that the core has already a packet prioritization algorithm, where type advert is bottom queue.

We are, but it's not related to the queue nor its prioritization.

[jbrazio]

I would personally prefer to keep the minimum allowed flood advert interval as 3 hours.

IMO 3h is too low as min flood, compromising and going into 6h min would represent already a 25% saving. Also 60m is too low for zero-hop, specially now with local discovery, zero-hop advert could even be considered obsolete.

[liamcottle]

IMO 3h is too low as min flood, compromising and going into 6h min would represent already a 25% saving. Also 60m is too low for zero-hop, specially now with local discovery, zero-hop advert could even be considered obsolete.

Depends on mesh size and what mesh operators want.

If @ripplebiz decides he wants to go ahead with an enforced minimum advert interval increase, I'd suggest that the minimum/maximum values are set as build flags, so they can be adjusted via the build script (for custom firmware distribution) without having to run a separate branch to allow lower minimums if mesh admins such as myself want 3 hour interval... 😂

I don't run official releases anyway, as I like to distribute firmware bins that have my radio preset baked in, so adding in a couple extra flags to set this would be my preferred way to go about it.

-D MIN_FLOOD_ADVERT_INTERVAL=3
-D MAX_FLOOD_ADVERT_INTERVAL=48

[ViezeVingertjes]

I don't run official releases anyway

Haha same! mine drops all adverts right when rx/tx util hits a certain threshold, but its not ideal and would be nicer to have this properly solved at some point. (not saying this is the final solution though, just gives a little bit more time to think of a better solution.)

[mattzzw]

zero-hop advert could even be considered obsolete.

I strongly disagree. All of the neighbourhood "discovery" can be done with zero hop adverts or requests.
E.g. when a repeater boots, it should only send a zero hop advert instead of a flood advert (I think I saw a pull request on that one recently).

The MC design philosophy is about data minimalism and robustness/reliability.

Automated flooding with packets of quite substantial size "for new user experience" is harming both.

[jbrazio]

zh-advert are obsolete because a flood advert will also notify zh-clients.
If I'm in a new zone that I need to know what repeaters are within reach right now.. I just discover them. (this was not possible in the past)

[wesselah]

zh-advert are obsolete because a flood advert will also notify zh-clients.
If I'm in a new zone that I need to know what repeaters are within reach right now.. I just discover them. (this was not possible in the past)

Sorry i do not agree with your statement that zero hop is obsolete. Why:

Meshcore is successfull and grows fast when a mesh doubles the number of advert triples. Its just numbers
Number of adverts = n*(n-1) which are the number of adverts that are generated or there must be something in meshcore that prevents this n = number of nodes .
At 500 nodes the number of adverts is 250.000 at every minimum advert time so at 6 hours the number of adverts is 1.000.000 adverts a day. When a mesh even growns bigger the number of adverts sent will exhaust even the dutycycle legally allowed.
So first off all flood adverts need to be restraint and controlled. The pr admitted is a big step forward but not sufficient.
Technics to contoll are:

1. TTL when used with the correct ttl number adjusted to the mesh size advert are controlable and keep there function
2. Sliding windows on advert based on repeater uptime the window becomes bigger and bigger with max to be determent.
3. Probabilistic flooding which is a change of a flood advert being forwarded or not .
4. Jitter which introduces a randon waiting time before flood advert will be forwarded normally 2-3 times the transmissiion time a normal packet need in the network.

So why do i not agree with you're statement that zero hop is obsolete. When flood adverts are minimized which is absolutely needed . Visibility off neighbors still need to be in place to give admins a way of checking link development and to check if links change due to network changes like changing cr or sf which will have impact on links. And zero hops is then the perfect solution already in place. And discovering neighbors is only bennefitting companions who are in a new surronding this does not function on an repeater.

I am an old network architect worked more than 40 years in networking not in rf networks but every network that needed flooding to give some insight died when it became to large. The network was overloaded with floods and an normal operation was impossible. That is in my opinion what is awaiting meshcore when it does not come in action and i support this pr because it is a first step. But please keep zerohop in place.

[ViezeVingertjes]

When a mesh even growns bigger the number of adverts sent will exhaust even the dutycycle legally allowed

Even without further mesh growth, we already exceed the legal duty cycle in the Netherlands during quiet periods; the MC default of 50% is four times higher than our 10% limit.

Visibility off neighbors still need to be in place to give admins a way of checking link development and to check if links change due to network changes like changing cr or sf which will have impact on links.

This is debatable: zero-hop adverts are already off by default, so the data is less reliable than it seems. Since these measurements are only snapshots, trace would be a far more reliable way to observe link changes. But you will need a way to discover them on the repeater though, so or it would need the same discovery method as companions, or zero hop will still be needed for that.

--

But i think we all agree, we need to eliminate automated flooding entirely, as it now accounts for over 50% of total traffic and overwhelms real messages with unwanted advertisements from across the country. On top of that, we also need better ways to limit companions that spam adverts manually, repeaters in boot-loops that flood advert every 30 seconds etc. Quite some challenges to overcome still.

[jbrazio]

@wesselah DISCOVER_REQ replaces zh-advert (in a much more efficient way), that was my only point on that sentence.

[wesselah]

@wesselah DISCOVER_REQ replaces zh-advert (in a much more efficient way), that was my only point on that sentence.

For companion only

[timdj]

@wesselah DISCOVER_REQ replaces zh-advert (in a much more efficient way), that was my only point on that sentence.

For companion only

can't be implemented on repeaters as well?

[ViezeVingertjes]

@wesselah DISCOVER_REQ replaces zh-advert (in a much more efficient way), that was my only point on that sentence.

For companion only

can't be implemented on repeaters as well?

That's would be the same as Meshtastic does now, which they havent solved as today and why its unusable.

From a pure congestion standpoint; we could simply allow it and above a certain airtime percentage, drop all flood adverts. That's how my repeater currently works and that seems to work fine, its always available for actual communication, even when people are in bootloops sending loads of adverts etc.

Then when its less busy and there is room for it, flood adverts are repeated again like normally. This way it doesnt interfere with eachother. But this is just one of many solutions, basically flood = bad.