Monitoring Power Query Online Memory And CPU Usage

2 years ago

source link: https://blog.crossjoin.co.uk/2022/06/26/monitoring-power-query-online-memory-and-cpu-usage/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

Monitoring Power Query Online Memory And CPU Usage

Power Query Online is, as the name suggests, the online version of Power Query – it’s what you use when you’re developing Power BI Dataflows for example. Sometimes when you’re building a complex, slow query in the Query Editor you’ll notice a message in the status bar at the bottom of the page telling you how long the query has been running for and how much memory and CPU it’s using:

The duration and CPU values are straightforward, but what does the memory value actually represent? It turns out it’s the “Commit (Bytes)” value documented here for Query Diagnostics, that’s to say the amount of virtual memory being used by the query. That’s different to the “Working Set (Bytes)” value which is the amount of physical memory used by the query, and which is not visible anywhere. For a more detailed discussion of these values in Power Query in Power BI Desktop see this post. The maximum commit or working set for a query evalation in Power Query Online isn’t officially documented anywhere (and may change) but I can say three things:

The maximum commit is larger than the maximimum working set.
If Power Query Online uses more than the maximum working set then query evaluation will get slow, so if your query uses a lot of memory (say, over 1GB – I suspect you’ll only see this message if it is using a lot of memory…) then you need to do some tuning to reduce it. Probably the best way to do this is to look at the query plan for your dataflow and try to avoid any operations marked as “Full Scan”, as documented here.
If your query uses more than the maximum commit then it may get cancelled and you’ll see an error (note that the maximum time a query evaluation can run for in Power Query Online anyway is 10 minutes, which is documented here).

[Thanks to Jorge Gomez Basanta for this information]

Like this:

Recommend

www.tuicool.com 6 years ago
Cache

Memory Usage of OpenWhisk Functions

Last week Red Hat introduced the open source framework Quarkus which promises fast boot times and low memory usages. This makes Quarkus interesting for Java wor...

www.mssqltips.com 4 years ago
Cache

SQL Server Memory Usage Query

SQL Server Memory Usage Query By: Eli Leiba | Updated: 2021-04-15 |

blog.appsignal.com 3 years ago
Cache

Monitoring Memory Usage for Erlang and Elixir Applications

In this post, we’ll do a quick overview of monitoring memory issues in Erlang and Elixir setups. We’ll do so by monitoring memory usage at three levels: Host, OS, and within the Erlang VM. Getting the Data To keep the article...

blog.jakubholy.net 3 years ago
Cache

Monitoring process memory/CPU usage with top and plotting it with gnuplot

Monitoring process memory/CPU usage with top and plotting it with gnuplot October 17, 2018

help.dreamhost.com 2 years ago
Cache

Monitoring DreamObjects usage and costs

Monitoring DreamObjects usage and costs Overview The Usage Report tab in the panel provides billing and storage details relating to your current DreamObjects service plan.