5

Remove Artificial Object Graph Limit in BinaryFormatter · Issue #23971 · dotnet/...

 3 years ago
source link: https://github.com/dotnet/runtime/issues/23971
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

Remove Artificial Object Graph Limit in BinaryFormatter #23971

Closed

Alois-xx opened this issue on Oct 27, 2017 · 14 comments

Comments

Contributor

Alois-xx commented on Oct 27, 2017

Since issue #20569 is solved we now can deserialize bigger object graphs with up to 12 million objects.
As I have found since a long time ago: https://aloiskraus.wordpress.com/2017/04/23/the-definitive-serialization-performance-guide/ there are issues with the max object count BinaryFormatter can cope with because it contains a hard coded prime number table which ends at 6584983 objects.

Please remove this artifical limit and do the same thing like e.g. in Dictionary which also needs prime numbers.

davidsh

changed the title Remove Artifical Object Graph Limit in BinaryFormatter

Remove Artificial Object Graph Limit in BinaryFormatter

on Oct 28, 2017

Member

joperezr commented on Nov 22, 2017

cc @ViktorHofer can you take a look?

will take a look this week.

BTW I notice the HashHelpers in CoreFX precomputes up to 2B (highest to fit in int) whereas the one in Corelib precomputes only up to 7M (and then generates primes). This means Dictionary (which uses the Corelib one) is generating a new prime after 7M entries. I suggest replacing the CoreCLR one with the CoreFX one.

Has for ObjectIDGenerator (for BinarySerializer) ideally it would use Hashhelpers but it chooses to start at 5 instead of 3 for some reason. maybe that could change.

@Alois-xx I enjoyed your blog post! Thanks for pointing out these issues.

Contributor

Author

Alois-xx commented on Nov 23, 2017

@danmosemsft thanks for the kind words. I always do a root cause analysis of interesting issues. This involves quite a bit of explorartory boundary testing which often leads to nice profiling graphs or Excel charts or both.

@Alois-xx what sort of problems are interesting to you in general? I could look out for an interesting perf optimization opportunity.

Contributor

Author

Alois-xx commented on Nov 27, 2017

@danmosemsft I work on a very large application which uses pretty much every .NET Framework (desktop) technology. Startup time is always an issue. Windows Forms/WPF interop is a source of pain with many temporary bitmaps beeing generated. The WPF measure calls to layout the UI controls are very costly if you have a large nested UI hierarchy.
The workflow foundation VB compile times and memory consumption are ludicrous. This is costing hundreds of MB for a 5 MB xaml file which is surprising to say the least. Despite having several long lasting calls for this issue nothing has happened since years.

From a .NET Core perspectivethe the costs of getting live data for processes (System.Diagnostics.Process ... ) seems to be overly costly which could be improved if I am e.g. only interesting in the memory footprint of other processes.

Would .NET Core improvements benefit you? Or is your goal that changes are stepping stone to port to .NET Framework?

For sure we are open to perf improvement suggestions or PRs for Process.

Contributor

Author

Alois-xx commented on Nov 27, 2017

edited

@danmosemsft I hope to get improvements done in .NET Core to light up in the regular .NET Framework as well which makes it very attractive to invest time right now since you are moving incredibly fast.
Switching over to .NET Core is currently due to the many dependencies to WPF/WinForms/WF ... not an option for some time. But for new projects .NET Core is definitely an attractive option if the web is the new UI.

@ViktorHofer I see you took this one... are you planning to do this for 2.1 (ie by Friday)?

Nice work

Contributor

Author

Alois-xx commented on Mar 30, 2018

@ViktorHofer, @danmosemsft , @jkotas : Awesome! The amount of dedication to arrive at the best solution is much appreciated.

@Alois-xx you are welcome, I hope you do end up basing new projects on .NET Core as you mention, and share other feedback with us as it comes up for you.

msftgits

transferred this issue from dotnet/corefx

on Feb 1

msftgits

added this to the 2.1.0 milestone

on Feb 1

msftbot bot

locked as resolved and limited conversation to collaborators

2 days ago

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Milestone

2.1.0

Linked pull requests

Successfully merging a pull request may close this issue.

6 participants

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK