Forums Tyrant

[SIM] iteratedecks 1.0.5 page 2

90 posts

Flag Post
Originally posted by MrHen:
Originally posted by Moraku:

I’m sure you’re already aware of potential problems on iterate-decks with battleground effects, but I recently received a PM from a concerned player where there appears to be a winrate conflict for their Copycat deck on the Fansite.

Yes; 1.0.2 will have a bunch of battleground bug fixes. It should be released tonight. In the case of Copycat, there was a bug where units were still Evading Mimic; would that have applied to their deck?

No, but the deck revolves around using Structures and Action cards with Mimic.

SimTyrant link

EDIT: I added a few more Copycat testcases to the repo to cover commanders, structures and action cards.

 
Flag Post

Output from iteratedecks 1.0.2:

iteratedecks-cli.exe TYCfCtu/fy+iu/fzCT+iBo -Q 4 -o -n 10000
Games won:          2602 /      10000
Games lost:         7373 /      10000
Games drawn:          25 /      10000

Output from SimTyrant:

Wins 2768 27.7% 
Losses 7207 72.1% 
Draws 25 0.3% 
Battles 10000 100.0% 

I can deal with a 1.6% difference. Looks like whatever issue there was got cleaned up with the bug fixes. 1.0.2 passes the Copycat tests you added.

 
Flag Post

1.0.3

  • Crush damage no longer triggers On Attack abilities
  • -a now uses achievement id instead of achievement index
  • Now supports more than 256 units at a time
 
Flag Post

I think that this simulator is “destroying” the winrates in fansite, for example:

28% for Pantheon Perfect
http://tyrant.40in.net/kg/raid.php?id=13#sdecks:did=14083

SimTyrant: 99.9% winrate.
Tyrant Optimize: 99.97% winrate.
iteratedecks: ??? I can’t run the sim, gives me an APPCRASH error.

 
Flag Post

Iterate-decks is just fine. Although, it might have something to do with the submission, rather than the simulator itself. Hard to tell which simulator is at fault until sss1 finishes implementing collision detection.

Games won:         99947 /     100000
Games lost:           19 /     100000
Games drawn:          34 /     100000
estimator=0.9995 confidence [0.9993;0.9996]; ANP=24.7823

 
Flag Post

http://tyrant.40in.net/kg/raid.php?id=13#sdecks:did=14105

Yet this deck does not seem to have the same problem. Hashing issue?

 
Flag Post
simulator Tyrant Optimizer
ver 1.0.1
submitter andor9
battles 1000000
battles won 1
anp 0.000080
date 1363138253
time simulating 100
win chance 0%

somehow test case got through from console. shouldn’t happen. should be fixed now.

 
Flag Post
Originally posted by hunterhogan:

http://tyrant.40in.net/kg/raid.php?id=13#sdecks:did=14105

Yet this deck does not seem to have the same problem. Hashing issue?

Dont know how but the Duncan deck and 10 Pyros now shows the corrent winrate, you cant find in the fansite because i deleted but you can add it and will show instantly the winrate stored in the fansite server.

 
Flag Post
Originally posted by DarkBlood1:

I think that this simulator is “destroying” the winrates in fansite, for example:
APPCRASH error.

Are you still having this problem? I assume you have downloaded the latest .xml data? If so, can you get me the version number output by —version? I cannot repoduce the problem on my end.

Obviously, if the app crashes it is not going to submit results to the Fansite.

 
Flag Post
Originally posted by sss1:

simulator Tyrant Optimizer

somehow test case got through from console. shouldn’t happen. should be fixed now.

That’s odd. That isn’t even the right simulator.

 
Flag Post
Originally posted by MrHen:
Originally posted by DarkBlood1:

APPCRASH error.

Are you still having this problem? I assume you have downloaded the latest .xml data? If so, can you get me the version number output by —version? I cannot repoduce the problem on my end.

Obviously, if the app crashes it is not going to submit results to the Fansite.

I forgot to update the XMLs, now I did it and everything works fine, sorry.

:shame:

 
Flag Post

1.0.4 released. You will need to update your .xml files to use cards from Worldship part 2.

  • Now supports Stun
  • Chaosed Jam now targets units that have already attacked
  • Blitzed units can now be Rallied
  • Mimicked Payback can now hit the mimicker
  • Chaosed abilities can now target already played units
 
Flag Post
Originally posted by MrHen:

Mimicked Payback can now hit the mimicker

what does this refer to?

mimicked strike can be paybacked onto the mimicker?

 
Flag Post

Yes. I worded the commit poorly. :P

 
Flag Post

I’ve been trying to figure out why my Burst skill achievement count (particularly for Misfire achievement) is different from both iterate-decks and tyrant-optimize. Correct me if I’m wrong, but is iterate-decks counting Burst for each time it misses a Flying assault?

 
Flag Post

seems the best place to answer MrHen suggestion would be here (to avoid offtopic in simTyrant thread)

what you are missing with grouping is – there are several possible factors that do cause each collision. just a few things for you to think about
- two decks for same topic can raise collision based on totally different bugs
- one bug can rise dozen collisions for one deck and only few for another (based on amount of simulations user were submitting – for first there could be 30 submissions by 50k, fnd for 2nd – only 2 with 1M)
- there isn’t a way to auto detect resons for bugs, the only possible grouping would be individual deck + submitter version; but then again – the reason collision was added can be different for two submissions even for 1 deck: submitter used fake data, average value for sim (win/anp) changed enough to rise collision (due to f.e. another sim invalidated part of results)

i still don’t understand the idea behind grouping at all. it’s not the tool for auto detecting what is wrong and where, but rather a tool to tell you “something can be wrong with this deck”, nothing more. i made that based on how i would be using it. open the page – find the largest collision deltas and look then one by one. it doesn’t matter whether there were 1k collisions for the deck or only 1 (that data doesn’t change a thing). the only thing matter is whether delta is 10% or 1%. maybe it’s not too convenient to have 100 collisions flooding first pages, but since they are all on first page, they are the most important ones. and once you’ll finish the bug, with only one click you will clear them all.

yes, i do understand that possibly by just looking on list of collision topics you could find out the possible bug for several topics, but you will have to check that anyway for each deck one by one (whether the real reason is the one you are thinking about).

as for a collisions fixed by other sims – there is calculated delta stored. once you’ll get to collisions caused and already fixed by another sim, simply looking for stored and current delta you’ll notice bug possibly already fixed. comparing average win/anp after each invalidation for possible collisions for OTHER sims on the topic to delete would be too much of unnecessary load. would require adding that check for each new submission as well – since it does affect that average value as well.

 
Flag Post
Originally posted by sss1:

- two decks for same topic can raise collision based on totally different bugs

Two different decks would mean two different scenarios.

- one bug can rise dozen collisions for one deck and only few for another (based on amount of simulations user were submitting – for first there could be 30 submissions by 50k, fnd for 2nd – only 2 with 1M)

Right; I would want to see two “groupings” then: One for Deck A and one for Deck B. I don’t care how many collisions were detected for each scenario; I also don’t care what the submission size was until after I start looking at all of the simulator data for that scenario.

- there isn’t a way to auto detect resons for bugs, the only possible grouping would be individual deck + submitter version; but then again – the reason collision was added can be different for two submissions even for 1 deck: submitter used fake data, average value for sim (win/anp) changed enough to rise collision (due to f.e. another sim invalidated part of results)

I am not looking for bug detection; that is my job as the developer. What I am looking for are scenarios that have different results between simulators. Even if there are two different types of collisions for one scenario, all I care about is the scenario because I will end up debugging them in the same manner. There is no way for me to triage collisions until after I look at the entire scenario anyway; it doesn’t do me any good to know how many collisions there actually were. The piece of information that was valuable is that there was any collisions.

i still don’t understand the idea behind grouping at all. it’s not the tool for auto detecting what is wrong and where, but rather a tool to tell you “something can be wrong with this deck”, nothing more.

Right; that’s what I want. But what I get right now is that same piece of information doubled because it is grouped by submission and not by deck. So if I have 2 submissions and each other simulator has 2 submissions, how many collision rows do I see? I want to see one.

i made that based on how i would be using it. open the page – find the largest collision deltas and look then one by one. it doesn’t matter whether there were 1k collisions for the deck or only 1 (that data doesn’t change a thing). the only thing matter is whether delta is 10% or 1%. maybe it’s not too convenient to have 100 collisions flooding first pages, but since they are all on first page, they are the most important ones. and once you’ll finish the bug, with only one click you will clear them all.

Actually, no, they are not the most important ones. The front page will have the collisions with the most unique bugs. It is virtually impossible to analyze a failure for an Achievement that is counting three different skills — especially when one of the skills itself is bugged instead of the proc counters. The most important collisions are the highest delta that points toward one unique bug. Those are the easiest to investigate and provide the biggest payoff.

But even if I were just scanning through to find generic problems instead of specific problems, I want to know how many scenarios are failing. Not how many collisions the top scenario has collected. The second, third, fourth collisions don’t add much information to what the first collision reported.

yes, i do understand that possibly by just looking on list of collision topics you could find out the possible bug for several topics, but you will have to check that anyway for each deck one by one (whether the real reason is the one you are thinking about).

No, actually, I won’t. What I do is look for a problem scenario and analyze it looking for the bug. Once the bug is fixed I rerun the scenario to verify it no longer collides and then I invalidate all of submission that would have been affected by the bug. When that happens, all of the topics that could have been effected by the bug are destroyed.

When I sit down to hunt bugs I go through the list of collisions triaging to find the highest priority. That priority does reflect the highest delta but I also skip over the scenarios that are likely to have multiple bugs until those are the only ones left.

as for a collisions fixed by other sims – there is calculated delta stored. once you’ll get to collisions caused and already fixed by another sim, simply looking for stored and current delta you’ll notice bug possibly already fixed.

What I’ll notice is an invalid collision. I will delete it and wait to see if a new collision appears for that scenario. There is no reason for me to investigate this scenario; if there is a bug it will show up. I will never investigate an old collision if I notice that (a) ID matches one of the other simulators and (b) the odd simulator out has since changed and now matches the other two.

comparing average win/anp after each invalidation for possible collisions for OTHER sims on the topic to delete would be too much of unnecessary load. would require adding that check for each new submission as well – since it does affect that average value as well.

Okay, so I think I am beginning to understand how the backend works more clearly. You are saying that for each scenario, an average of “all simulators but this one” is compared against each incoming submission? (Or possibly, whenever the average for one simulator gets updated with a new submission it rechecks the average a collision?)

Personally, I won’t care about that collected average once I know it has changed. I don’t even want to look at the collision. Can I at least get an “invalid” or “outdated” bit that denotes the underlying numbers have since changed?

Or how about a way to manually trigger a recalculate for the collision? If I am understanding correctly, an incoming submission for Simulator A will not cause a collision in Simulator B?

In the end, I don’t want to create more work for you. Without knowing how the backend works, having multiple collisions for the same scenario is useless to me. I either skip over them or stop at the first one and fix the bug. If that just isn’t possible (or you just don’t agree with me) I can live with it. I don’t personally understand the value of having it show up more than once.

A concrete example — the top three collisions currently in my list are:

A and C are the exact same scenario. I don’t need two links to the same problem. Even if I wanted to see the submission breakdown:

Same link, two rows. Where is the value in having that problem show up in two different rows?

Row 4 is unique, row 5 is a duplicate of row 2. Row 9 is a duplicate of row 4. And so on.

 
Flag Post
Originally posted by MrHen:

Okay, so I think I am beginning to understand how the backend works more clearly. You are saying that for each scenario, an average of “all simulators but this one” is compared against each incoming submission? (Or possibly, whenever the average for one simulator gets updated with a new submission it rechecks the average a collision?)

yes, that’s how it’s working at the moment. except the second part – it’s not rechecked once average is updated.

Personally, I won’t care about that collected average once I know it has changed. I don’t even want to look at the collision. Can I at least get an “invalid” or “outdated” bit that denotes the underlying numbers have since changed?

10 more params for date, allowed delta of delta for delta for auto removing collisions? or just a notify that collision is suspected as resolved (if % chance or anp delta is beyond 5%/1 compared to the moment of check)

Or how about a way to manually trigger a recalculate for the collision? If I am understanding correctly, an incoming submission for Simulator A will not cause a collision in Simulator B?

i doubt you can’t say in 5 seconds after looking submissions page that something has changed. even i can do that.

ok. it seems you are having all this collision stuff way too close :) i’ll look at what can be added, but there would be 1 limit though to avoid caching too big keys. 1k collisions viewed/paged at one moment. unless you clear these, you will not be able to look next ones. would that be more helpful?

 
Flag Post

small note about “collision grouping” idea: I often find myself browsing through each page to find collisions from SimTyrant itself, as that implies there are things I forgot to invalidate, or I’ve created a new bug I was unaware of. There are also too many mixed bugs with iterate-decks, that I decided to look for tyrant-optimize collisions instead.

 
Flag Post
Originally posted by Moraku:

small note about “collision grouping” idea: I often find myself browsing through each page to find collisions from SimTyrant itself, as that implies there are things I forgot to invalidate, or I’ve created a new bug I was unaware of. There are also too many mixed bugs with iterate-decks, that I decided to look for tyrant-optimize collisions instead.

not sure what you need here:
- delete all collisions …
- do not rise collisions …
- hide all collisions …
where “…” is “with the sim X”

 
Flag Post
Originally posted by sss1:
Originally posted by Moraku:

small note about “collision grouping” idea: I often find myself browsing through each page to find collisions from SimTyrant itself, as that implies there are things I forgot to invalidate, or I’ve created a new bug I was unaware of. There are also too many mixed bugs with iterate-decks, that I decided to look for tyrant-optimize collisions instead.

not sure what you need here:
- delete all collisions …
- do not rise collisions …
- hide all collisions …
where “…” is “with the sim X”

I think a sort function would have sufficed. I noticed that the collisions are currently sorted by delta.

 
Flag Post
Originally posted by sss1:

10 more params for date, allowed delta of delta for delta for auto removing collisions? or just a notify that collision is suspected as resolved (if % chance or anp delta is beyond 5%/1 compared to the moment of check)

Without knowing the details of your backend I couldn’t say. My first guess would have been, “Simulator A just invalidated Scenario X; flag all collisions on Scenario X.” But that assumes you have a quick way to find and update collisions based on “scenarios.”

The goal is just to prevent me from ever clicking on a collision to view the details only to find the collision is stale because one of the other simulators fixed a bug.

i doubt you can’t say in 5 seconds after looking submissions page that something has changed. even i can do that.

Yeah, you’re right. That one didn’t make much sense.

ok. it seems you are having all this collision stuff way too close :) i’ll look at what can be added, but there would be 1 limit though to avoid caching too big keys. 1k collisions viewed/paged at one moment. unless you clear these, you will not be able to look next ones. would that be more helpful?

Sure. To be honest, I have an opinion about this but won’t cry if I don’t get it. If this is a major hassle let it be. Like you said in a different discussion, the only real reason this is an issue worth talking about is because of the initial collision load.

The other side of this, by the way, is that I don’t really care if the grouping happens in the database. If the collisions page just ran JavaScript to group/hide/collapse the collisions that had identical links it would do exactly what I want. I find it surprising that you can delete/ignore similar scenarios already but aren’t able to “hide” similar scenarios.

Pseudocode:

  • IF collision deck link already on page, print as collapsed
  • ELSE print collision information

I wonder if CSS can just “do” this… is there a, “hide all elements with class X except the first” option?

 
Flag Post

I find it surprising that you can delete/ignore similar scenarios already but aren’t able to “hide” similar scenarios.

it’s one update sql (by deckId) compared to select of 1000 rows, sorting, grouping, summing and caching the result + invalidate the cache and repeat each time you invalidate/delete any of the collision.

Pseudocode:

  • IF collision deck link already on page, print as collapsed
  • ELSE print collision information

I wonder if CSS can just “do” this… is there a, “hide all elements with class X except the first” option?

100 pages with only 1 collision on each. is that really what you want? because that’s what you’ll get :D

 
Flag Post
Originally posted by sss1:

100 pages with only 1 collision on each. is that really what you want? because that’s what you’ll get :D

That’s basically what I am getting now… ;)

In reality, I see about a dozen or more unique issues per page which is fine. I just don’t want to see the duplicates because it makes it harder to skim through looking for the high priority scenarios.

But yeah, getting an “ignore similar scenarios but keep this one” button would do exactly what I want even if all it does is hide duplicates on the currently visible page. It actually did whatever you do for the current delete/ignore but kept still kept the selected one it would be perfect.

As for the difference between delete/ignore and hide from the database side of things, yeah, that makes sense. I won’t bother trying to armchair design your database. You know what you are doing and I’ll just end up making a fool of myself by suggesting random, crappy ideas. :)

 
Flag Post

added draft grouping stuff for collisions. will look at filtering based on the sim collided against a little bit later.

Step 24 (Genesis) 51 / 52

52 simulation collision for 51 deck. something is definetelly working different in iterateDecks.

as a side note:
99.9999% vs 100%, what is that 0.0001 iterateDecks found and others failed to with 2M+ battles?