Remove Artificial Object Graph Limit in BinaryFormatter · Issue #23971 · dotnet/...
source link: https://github.com/dotnet/runtime/issues/23971
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Remove Artificial Object Graph Limit in BinaryFormatter #23971
Closed
Alois-xx opened this issue on Oct 27, 2017 · 14 comments
Comments
Contributor
Alois-xx commented on Oct 27, 2017
Since issue #20569 is solved we now can deserialize bigger object graphs with up to 12 million objects.
As I have found since a long time ago: https://aloiskraus.wordpress.com/2017/04/23/the-definitive-serialization-performance-guide/ there are issues with the max object count BinaryFormatter can cope with because it contains a hard coded prime number table which ends at 6584983 objects.
Please remove this artifical limit and do the same thing like e.g. in Dictionary which also needs prime numbers.
changed the title Remove Artifical Object Graph Limit in BinaryFormatter
Remove Artificial Object Graph Limit in BinaryFormatter
Member
joperezr commented on Nov 22, 2017
cc @ViktorHofer can you take a look?
will take a look this week.
BTW I notice the HashHelpers in CoreFX precomputes up to 2B (highest to fit in int) whereas the one in Corelib precomputes only up to 7M (and then generates primes). This means Dictionary (which uses the Corelib one) is generating a new prime after 7M entries. I suggest replacing the CoreCLR one with the CoreFX one.
Has for ObjectIDGenerator (for BinarySerializer) ideally it would use Hashhelpers but it chooses to start at 5 instead of 3 for some reason. maybe that could change.
@Alois-xx I enjoyed your blog post! Thanks for pointing out these issues.
Contributor
Author
Alois-xx commented on Nov 23, 2017
@danmosemsft thanks for the kind words. I always do a root cause analysis of interesting issues. This involves quite a bit of explorartory boundary testing which often leads to nice profiling graphs or Excel charts or both.
@Alois-xx what sort of problems are interesting to you in general? I could look out for an interesting perf optimization opportunity.
Contributor
Author
Alois-xx commented on Nov 27, 2017
@danmosemsft I work on a very large application which uses pretty much every .NET Framework (desktop) technology. Startup time is always an issue. Windows Forms/WPF interop is a source of pain with many temporary bitmaps beeing generated. The WPF measure calls to layout the UI controls are very costly if you have a large nested UI hierarchy.
The workflow foundation VB compile times and memory consumption are ludicrous. This is costing hundreds of MB for a 5 MB xaml file which is surprising to say the least. Despite having several long lasting calls for this issue nothing has happened since years.
From a .NET Core perspectivethe the costs of getting live data for processes (System.Diagnostics.Process ... ) seems to be overly costly which could be improved if I am e.g. only interesting in the memory footprint of other processes.
Would .NET Core improvements benefit you? Or is your goal that changes are stepping stone to port to .NET Framework?
For sure we are open to perf improvement suggestions or PRs for Process.
@danmosemsft I hope to get improvements done in .NET Core to light up in the regular .NET Framework as well which makes it very attractive to invest time right now since you are moving incredibly fast.
Switching over to .NET Core is currently due to the many dependencies to WPF/WinForms/WF ... not an option for some time. But for new projects .NET Core is definitely an attractive option if the web is the new UI.
@ViktorHofer I see you took this one... are you planning to do this for 2.1 (ie by Friday)?
Nice work
Contributor
Author
Alois-xx commented on Mar 30, 2018
@ViktorHofer, @danmosemsft , @jkotas : Awesome! The amount of dedication to arrive at the best solution is much appreciated.
@Alois-xx you are welcome, I hope you do end up basing new projects on .NET Core as you mention, and share other feedback with us as it comes up for you.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
None yet
Successfully merging a pull request may close this issue.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK