With the previous approach we'd require at least an additional pointer
per item to store them in a list (15-18% increase in the overhead per
item). Instead we switch from handling collisions with overflow lists to
an open addressing scheme and store the actual table as variable-sized
indices pointing into an array of all inserted items in their original
order.
This can reduce the memory overhead even compared to the previous
implementation (especially for smaller tables), but because the array for
items is preallocated whenever the table is resized, it can be worse for
certain numbers of items. However, avoiding all the allocations required
by the previous design is actually a big advantage.
Depending on the usage pattern, the performance can improve quite a bit (in
particular when inserting many items). The raw lookup performance is a bit
slower as probing lengths increase with open addressing, but there are some
caching benefits due to the compact storage. So for general usage the
performance should be better. For instance, one test I did was counting the
occurrences of words in a list of 1'000'000 randomly selected words from a
dictionary of ~58'000 words (i.e. using a counter stored under each word as
key). The new implementation was ~8% faster on average while requiring
10% less memory.
Since we can't remove items from the array (would change the indices of all
items that follow it) we just mark them as removed and remove them once the
hash table is resized/rehashed (the cells in the hash table for these may
be reused). Due to this the latter may also happen if the number of stored
items does not increase e.g. after a series of remove/put operations (each
insertion requires storage in the array, no matter if items were removed).
So if the capacity is exhausted, the table is resized/rehashed (after lots
of removals the size may even be reduced) and all items marked as removed
are simply skipped.
Compared to the previous implementation the load factor/capacity is
lowered to reduce chances of collisions and to avoid primary clustering to
some degree. However, the latter in particular, but the open addressing
scheme in general, make this implementation completely unsuited for the
get_match() functionality (purposefully hashing to the same value and,
therefore, increasing the probing length and clustering). And keeping the
keys optionally sorted would complicate the code significantly. So we just
keep the existing hashlist_t implementation without adding code to maintain
the overall insertion order (we could add that feature optionally later, but
with the mentioned overhead for one or two pointers).
The maximum size is currently not changed. With the new implementation
this translates to a hard limit for the maximum number of items that can be
held in the table (=CAPACITY(MAX_SIZE)). Since this equals 715'827'882
items with the current settings, this shouldn't be a problem in practice,
the table alone would require 20 GiB in memory for that many items. The
hashlist_t implementation doesn't have that limitation due to the overflow
lists (it can store beyond it's capacity) but it itself would require over
29 GiB of memory to hold that many items.
The main intention here is that we can change the hashtable_t
implementation without being impeded by the special requirements imposed
by get_match() and sorting the keys/items in buckets.
This can improve negative lookups, but is mostly intended to be used
with get_match() so keys/items can be matched/enumerated in a specific
order. It's like storing sorted linked lists under a shared key but
with less memory overhead.
They are not marked as temporary addresses so make sure we always return
them whether temporary addresses are preferred as source addresses or not
as we need to enumerate them when searching for addresses in traffic selectors
to install routes.
Fixes: 9f12b8a61c ("kernel-netlink: Enumerate temporary IPv6 addresses according to config")
We don't track CHILD_SA down events anymore and rely on NM's initial timeout
to let the user know if the connection failed initially. So we also don't
have to explicitly differentiate between initial connection failures and
later ones like we do an Android. Also, with the default retransmission
settings, there will only be one keying try as NM's timeout is lower than
the combined retransmission timeout of 165s.
There is no visual indicator while the connection is reestablished later.
Fixes#3300.
If a MOBIKE task is deferred, the retransmission counter is reset to 0
when reinitiating. So if there were retransmits before, this alert would
not be triggered if a response is received now without retransmits.
We usually have a local IP already via ike_sa_t::resolve_hosts() before
build_i() is called but if that's not the case, it's more likely we have
one after we processed the first response (it might also have changed).
There is a potential chance we still don't have one if the socket API
doesn't provide us with the destination address of received messages,
but that seems not very likely nowadays.
We need the PSK/identity already when deriving the keys in process_i().
Fixes: 1665a4e050 ("ikev1: Use actual local identity as initiator or aggressive mode responder")
The native parseInetAddressBytes() method called by that method is not
available when running the tests.
Not very pretty and there are some warnings because PowerMock does
reflection in some illegal way but it fixes the unit tests and does
not require any new dependencies like Apache Commons or Guava just to
parse IP addresses without DNS lookup.
Fixes: 2ef473be15 ("android: Use helper to parse IP addresses where appropriate")
Fixes#3443.
Starting with Android 6, the system will aggressively suspend apps when
the device is idle (Doze mode). With Android 10 on a Pixel 4 this seems
to happen after about 70 minutes. Then the scheduler thread in our
default scheduler is only woken rarely, combined with our previous use
of the monotonic clock it meant that events were executed with severe
delays and noticing that there was such a delay. This was particularly
bad in regards to NAT keepalives as it usually meant that the device was
not reachable anymore from the outside.
Some changes here try to improve that situation, e.g. the clock is switched
to CLOCK_REALTIME (Bionic doesn't support CLOCK_BOOTTIME for condvars) so we
can measure the actual difference e.g. since the last outbound message,
other changes try to ensure that connectivity is restored after being asleep
for a while (send DPD instead of keepalive after a long delay, send DPD even
if path to peer stays the same).
However, the most significant change is the replacement of the default
scheduler with one specifically designed for Android. It schedules
long-term events via AlarmManager, which allows waking up the app even
if the system put it to sleep. The latter requires adding the app to the
system's battery optimization whitelist, which is requested from the
user automatically if necessary. With this, NAT keepalives and rekeyings
are now scheduled accurately, with little changes to the battery usage.
If the app is not whitelisted (there is a setting to ignore this), events
are delayed by up to 15 minutes after about 70 minutes, so behind a NAT
the device won't be reachable from the outside afterwards (connectivity
should be restored as soon as the device is woken from deep sleep by the
user).
Fixes#3364.
This allows users to ignore whether the app is on the device's power
whitelist without a warning. The flag is currently not set
automatically if the user denies the request.
This is necessary so we can actually schedule events accurately in Doze
mode. Otherwise, we'd only get woken in intervals of several minutes (up to
15 according to the docs) after about an hour.
This uses AlarmManager to schedule events in a way that ensures the app
is woken up (requires whitelisting when in Doze mode to be woken up at
the exact time, otherwise there are delays of up to 15 minutes).
Retransmission jobs for old requests for which we already received a
response previously left the impression that messages were sent more
recently than was actually the case.
task_manager_t always defined INVALID_STATE as possible return value if
no retransmit was sent, this just was never actually returned.
I guess we could further differentiate between actual invalid states
(e.g. if we already received the response) and when we don't send a
retransmit for other reasons e.g. because the IKE_SA became stale.
Previously, if the two utility functions were called while the VPN
connection was established (i.e. charon was initialized) the logger for
libstrongswan would get reset to the initial log handler. So certain
log messages would not get logged to the log file after the TUN device
was created (one of the helpers is used to convert IPs there).
A new NAT mapping might be created even if the IP stays the same. Due to
the DPD fallback with NAT keep-alives this might only be necessary in
corner cases, if at all.
This is useful on Android where the app might not be able to send
keep-alives if the device is asleep for a while. If the NAT mapping
has been deleted in the mean time, the NAT-D payloads allow detecting
this and connectivity can be restored by doing a MOBIKE update or
recreating the SA if the peer already deleted it because the client
wasn't reachable.
This allows measuring the delay between events more accurately if a
device is often suspended.
While CLOCK_BOOTTIME would be preferable, Android's bionic C library
does not support it for condvars.
On some systems it might be preferable to use e.g. CLOCK_BOOTTIME
instead of CLOCK_MONOTONIC, which is also not affected by time
adjustments but includes times when the system was suspended.
XML resources are apparently not supported there. Moving the icon to
the mipmap folders should fix that. Aliases are defined for the icons on
Android < 8.0.
Tobias Brunner