Parallel processing in Dynamics NAV

Parallel processing. When we hear that word we think of C# to make that magic happen.

But can we use C/AL for that kind of magic?

No? Think again!

YES! And without .NET tricks! Using pure C/AL!

But do we need it? Well, most of the time we don’t need it at all. After all, it is an ERP. And ERP do short and (hopefully) fast transactions so, yes, we don’t need parallel processing for that.

So when would we need it in NAV?

Sometimes we need a report that needs to read a lot of data and does a lot of processing on it.

An example:

You have a report that:

1. Needs to read all G/L Entries and do some calculations on it like totalising them.
2. Also needs to read all Item Ledger Entries and do some calculations on it like totalising them.
3. Also needs to read all Value Entries and do some calculations on it like totalising them.

Let’s suppose that each of them takes 10 minutes to read and process. That makes 30 minutes of processing. And the service tier is almost doing nothing because it is only 1 session that does the work. This means that session is CPU-core-bound. And SQL Serving is also doing almost nothing because it is waiting for the NAV service tier to consume the data it wants to send to the NAV service tier.

So wouldn’t it be nice if we could run those 3 parts in parallel?

It might not be a good idea to do that during the most busy hours because you might shift the slow-factor from purely CPU-core-bound to NAV-server-memory-bound or SQL Server-capabilities bound or also CPU-bound (having all cores of all CPU’s of the NAV-server to 100%). This would slow down all other operations.

How does it work?

We can do that with the STARTSESSION-command. But won’t we lose time saving the calculated data to disk so the master session can read it? Yes, we would. So we need a better way.

So how can we pass the data from the background session to the master session (or the other way around)?

When you run STARTSESSION, you can pass a record like calling any codeunit passing a record to it.

But did you know that the record can be a temptable? And that you can pass data from the master session to the background session and also back?

You can run as many background sessions as you want (well, I did a small test with starting sessions with loops in it and sleeping most of the time. And I have started 200 without any problems. But probably because they weren’t doing anything.).

So basically what it does is this:

Start all your sessions and save the session ID that NAV gives back in a temptable.

Your main session needs to wait until all other sessions have finished. You do that by checking the “Active Session” table until none of the started sessions still exist in it anymore.

Now your main session will have all data in the temporary tables. So it can do its final processing like putting all the data together.

A small warning:

I did try with having only 1 temptable and using the same temptable in all background sessions and it is effectively using the same temptable in all background sessions. But when I started testing the performance, I noticed that temporary records were lost somewhere in cyberspace and I also could crash the NAV service tier quite easily.

Probably it is because of the implementation of the temptables in NAV. It does not support that different processes are writing to the same temptable structure. Maybe Microsoft will fix that in a future version, but I doubt it.

PARALLEL PROCESSING 20160410.txt (had to rename it to .doc. WordPress doesn’t seem to support .txt……..)