TheMaximalBeing made benchmark tests on various AI commands back in 2020 to test their overall impact on game performance and posted them in the AI Scripters forums here: Link. Knowing the impact of these commands on performance can help scripters avoid lag. In DE, any script pass that lasts less than 20 milliseconds will not cause lag, and you can use up-get-precise-time to track script pass durations. Here is the article TheMaximalBeing posted about command performance:
Hi everyone,
For a long time, I've suspected that some of my DUC loops might be a bit slow. But just how slow are they? How many times can you issue a command before it causes a noticeable lag? I've decided to test this objectively!
For these tests I set up a loop over a command of interest. The loop was timed in *real* milliseconds, and i also found the time between rule passes (also in real milliseconds). I measured how long the loop took as a percentage of total time between rule passes.
Then the number of loops was increased until it took roughly 50% of the time between consecutive rule passes. At this point there is an obvious lag. The number of loops it took to reach this point for each command + setup is listed below.
All testing done on fast speed, WK. If you know of some other commonly used command + setup that could be slow but is not in the table then please let me know.
Rule of Thumb: Make sure you are doing less than 1/10 of the number of iterations for these commands in each pass of your AI (compared to the numbers in the table below).
| # | Description | Setup | Loops for 50% Lag | Speed |
|---|---|---|---|---|
| 0 | Empty Loop | Just an empty loop body | 11,000,000 | Very Fast |
| 1 | Chat Messages (chat-local-to-self) | Chat to self | 100 * | Very Slow |
| 2A | Basic DUC Search using localList (up-find-local) | Find 240 non-existent units Only 1 unit on map for each player | 4,500,000 | Very Fast |
| 2B | Find 240 non-existent units 400 enemy units on map Only 1 self unit exists | 4,500,000 | Very Fast | |
| 2C | Find 240 non-existent units 400 of my units on map | 300,000 | Fast | |
| 2D | Find 1 non-existent unit 400 of my units on map | 300,000 | Fast | |
| 2E | Find 1 existent unit 400 of them on map and nothing else | 800,000 | Fast | |
| 2F | Find 240 existent units 400 of them on map and nothing else | 80,000 | Medium | |
| 2G | Find 40 existent units 400 my units on map | 400,000 | Fast | |
| 2H | Find 240 existent units by class rather than id 400 of my units on map | 80,000 | Medium | |
| 3A | Filtered DUC Search using localList (up-find-local) | Find 240 non-existent units Filter distance (up-filter-distance) 400 my units on map | 300,000 | Fast |
| 3B | Find 240 existent units Filter distance 400 my units on map No units in radius | 50,000 | Medium | |
| 3C | Find 240 existent units Filter distance 400 my units on map All units in radius | 80,000 | Medium | |
| 3D | Find 240 existent units Filter include with cmd-id set to find only military (up-filter-include) | 80,000 | Medium | |
| 4A | Control Units with DUC (up-target-point) | Command 1 unit to move at random | 600,000 | Fast |
| 4B | Command 240 units to move at random individually | 400 units ** | Very Slow | |
| 5A | Removing Stuff from LocalList (up-remove-objects) | list size 240 remove base-type (no matches) | 250,000 | Fast |
| 5B | list size 40 remove base-type (no matches) | 1,300,000 | Very Fast | |
| 5C | list size 240 remove exact distance (no matches) | 120,000 | Fast | |
| 5D | list size 240 remove exact precise-x (no matches) | 200,000 | Fast | |
| 5E | list size 240 remove base-type all matches find inside loop | 50,000 | Medium | |
| 6A | Sorting the LocalList (up-clean-search) | Sort list size of 240 2 different object-data's Uncorrelated object-data | 3,000 (x2) | Slow |
| 6B | list size 240 Sort and unsort same object-data | 7,000 (x2) | Slow | |
| 6C | list size 40 Sort and unsort Uncorrelated object-data | 45,000 (x2) | Medium | |
| 6D | list size 20 Sort and unsort Uncorrelated object-data | 300,000 (x2) | Fast | |
| 6E | list size 5 Sort and unsort Uncorrelated object-data | 1,000,000 (x2) | Very Fast | |
| 7A | DUC search using RemoteList (up-find-remote) | Find 40 enemy units undiscovered but existent Enemy has 400 units | 4,000,000 | Fast |
| 7B | Find 40 enemy units discovered and existent Enemy has 400 units | 400,000 | Fast | |
| 7C | Find 40 enemy units Enemy has 0 units | 4,000,000 | Fast | |
| 7D | Find 40 enemy units discovered and existent Enemy has 40 units | 500,000 | Fast | |
| 7E | Find 40 ally units discovered and existent Ally has 400 units | 400,000 | Fast | |
| 8 | Loading a stored list back into RemoteList (up-set-group) | Group size = 40 Loop over(up-set-group) to set the RemoteList | 5,000,000 | Very Fast |
| 9A | Pathing (up-get-path-distance) | Pathing distance for 1 unit to point far away (200 tiles) No obstacles | 6,000 | Slow |
| 9B | Pathing distance for 1 unit to point far away (200 tiles) Lots of obstacles | 4,500 | Slow | |
| 9C | Pathing distance for 1 unit to point far away (200 tiles) Path does not exist | 4,500 | Slow | |
| 9D | Pathing distance for 1 unit to point nearby (3 tiles) No obstacles | 1,000,000 | Very Fast | |
| 9E | Pathing distance for 1 unit to point extremely far (340 tiles) No obstacles | 1,700 | Slow | |
| 10A | Building at points | (up-can-build-line) For palisade-wall Illegal point | 1,700 | Slow |
| 10B | (up-can-build-line) For palisade-wall Legal point | 1,400,000 | Very Fast | |
| 10C | (up-build-line) For palisade-wall Illegal point | 1,700,000 | Very Fast | |
| 10D | (up-build-line) For palisade-wall Legal point | 240 tiles placed *** | Very Slow | |
| 11 | Object Data (any unit) (up-get-object-type-data) | All IDs in the range 0 to 1400 tested unit lines tested Several object-data types tested | 5,000,000 | Very Fast |
| 12 | Creating / clearing search Groups (up-create-group) (up-reset-group) | Started with 40 militia in local list. Then repeatedly created and reset group of 40 units (GroupId = 0). | 4,000,000 | Very Fast |
| 13 | Setting the group flag (up-modify-group-flag) | Same as above but also setting the Ctrl group flag each time. | 1,700,000 | Very Fast |
| 14 | DUC unit training (up-target-point) | Using up-target-point with action-train to train militia. Local list consisted of 240 barracks, so there were 240 militia trained each for each train command. Then this whole process was repeated in a loop. Then cancelling them with up-reset-building. | 60,000*** | Medium |
| 15 | Getting map point contents (up-get-point-contains) | Scanning at random map points to see what unique IDs are located there. Tested with all-units-class (-1) and with several units. | 1,500,000 | Very Fast |
| 16A | DUC unit movement (up-target-point) | Moving groups of units to map border (group size = 1) | 400 groups **** | Very Slow |
| 16B | Moving groups of units to map border (group size = 5) | 400 groups **** | Very Slow | |
| 16C | Moving groups of units to map border (group size = 20) | 400 groups **** | Very Slow |
Test0: The main point of this test was to check the point at which the bottleneck is the for-loop itself. None of the other tests got too close to this point.
Test1: This test checked the performance of chat messages, which turned out to be extremely poor. The lag is clearly outside the rule pass so it's hard to quantify exactly the 50% point. 100 chats per rule pass was quite slow. Players who can't see the messages are not affected.
Test2: These tests were designed to test the performance of DUC searches without any filters. We can see that the performance of up-search-local is only affected by YOUR units and depends on the number of units you have. Though once the list is full (or meets the number of requested units) it may stop the search early. If you try to find a unit which does not exist, then it will have to check every unit to prove it doesn't exist. On the other hand, if the unit is abundant the search may find all the units you requested early - which is why the case for searching for 1 existent unit is way faster than 1 non-existent unit.
Test3: These tests where designed to check whether adding filters affected the performance of DUC searches. If the units didn't exist in the first place, then the filters clearly do not affect performance. But event in cases where the units did exist and the filter requirements were met, then there was no difference in performance compared to the no filter case. Clearly performance it limited by adding units to the list rather than the filter.
Test4: This time we control the units by targeting a point on the map. Interestingly, controlling the same unit many times did not cause poor performance. I don't think the game actually controls them until after the AI loop, and in this case, it probably only uses the most recent command (it just overrides it each time in the AI loop). Controlling many units does cause poor performance though. Again, the Lag is outside the rule pass itself. I found it quite laggy when moving 240 units randomly around them map (each controlled individually). In this case the lag is probably caused by path calculations.
Test5: These tests were designed to test the performance of up-remove objects. The first 3 tests didn't actually remove stuff from the list. In cases where you don't need to remove much, the performance depends on the particular object-data you use. Removing ALL units from the list each time looks slower, but in this case the performance is actually limited by the fact that I had to refill the list each loop.
Test6: These tests were designed to test the performance on sorting the lists. I had to do two sorts per loop on two different object-data's so that we are not sorting an already sorted list. Sorting algorithms usually have an n.log(n) complexity, so we expect that halving list size yields more than double the performance. This was clearly the case here - sorting 20 units is quite fast while 240 units is extremely slow.
Test7: This time we do DUC searches on the remote list. These results must be interpreted carefully. The AI keeps a list of known units for each player. If it hasn't discovered the units then it doesn't need to search through them. Performance is the same as for the local list if we are doing an identical search.
Test8: Rather than looping over an identical search (assuming that we need to do this), we might want to use up-create-group and in the loop up-set group to avoid searching again. This clearly leads to a massive performance boost.
Test9: We test up-get-path-distance. Interestingly, obstacles on the map only slightly affected the performance (even a maze had little effect!). Instead the performance is primarily determined by how far away the object must move - and the difference can be huge!
Test10: This time we test the performance of up-can-build-line and up-build line at a specific point. The up-build line check was extremely fast, as was up-build-line whenever it can't actually build it (it probably does the check first). But actually building them is a bit more complex. I found that building 240 palisades in a single rule pass was a bit slow - but the performance primarily depended on how many foundations had already been placed. The first rule pass was in fast quite fast, but it gets slower each time. After 8 rule passes or so (with 8*240 foundations) it was almost a standstill. The lag was outside the rule pass.
Test11: This test was designed to test the performance of "up-get-object-type-data" (as requested by Marathon). First the data-type was set to "object-data-train-site" and a variety of units and unit-lines were tested. It turns out that all units gave close to the same performance. I also ran another test to pick out the slowest performing unit ID in the range 0 to 1400 but it turns out that there was negligible difference in performance. Note that invalid unit IDs simply return -2. I also tested this with other data including "object-data-train-time", "object-data-base-attack" and "object-data-hitpoints" and all gave identical performance. In all cases the command is very fast.
Test12: The test was designed to test the performance of creating search groups. The performance is very good and we should not have to worry about this command.
Test13: This test was designed to test the performance of setting the group flag. The performance is very good and we should not have to worry about this command.
Test14: This test was designed to test the performance of unit training via DUC. The format of this command looks like:
(up-target-point 0 action-train typeOp inOpUnitId)
This will request that the unit be trained by all objects in the local-list. Note that it cannot queue units beyond the housing cap. But it does ignore sn-enable-training-queue. So it is allowed to keep queueing until the queue is full (15 units) or until you have reached the housing cap.
To test performance, I set the pop-cap to 1000 and trained militia in 240 barracks simultaneously (with 240 barracks in the local-list). This was done in a loop. The AI could comfortably train 1000 units in a single rule pass and we can only make the command lag if we try to train ~60000 units, in which case 59000 are blocked. In practise this command should never cause poor performance.
Test15: This command was designed to test the performance of "up-get-point-contains". It turned out to be surprisingly fast. I originally thought it might be looping over a large number of units. But it seems more likely that the game is storing the contents of each tile somewhere. You should be able to scan tiles without ever having to worry about performance.
Test16: This test was designed to test the performance of moving units in groups rather than individually. I wanted to know if moving say 20 units as a group is faster than moving them individually. It turned out that moving them in large groups was much faster but the reason is not so obvious.
From my testing, it became apparent that rather than processing all of the unit move commands straight away, the game is storing them in a queue. When a new move command is issued the unit will immediately stop. Then it must wait until its new move command gets processed. If you do too many move commands in a single rule pass (say 400) then there will be a delay of around 3" before some units start moving.
The units that were put into the start of the queue will move straight away, and the delay gradually increases as you move further down the queue.
This make performance hard to test. The command won't actually cause poor performance - instead units will become unresponsive because the queue is getting too long. If you try to move say 400 units each rule pass then many units will stutter (except the lucky ones at the start of the queue).
It seems that the game processes unit movement per group rather than per unit. So, it's much more responsive if you move your units in large groups rather than individually. Its best if you see this for yourself. I've uploaded a demo scenario and 3 scripts below which move units in groups towards the edge of the map every 2". You will see that the stutter is much worst when we try to move them individually.
Link to DUC Movement Test scenario and AI scripts used for Test 16: DUCmoveTest.zip
I am actually surprised by how good the performance of DUC is. Honestly, I've probably been worrying about it too much. It's generally OK to loop over a DUC search, even a double loop (of say 40 x 40) is OK in many cases. Although watch out for the things marked slow / very slow. Some ways to improve include: