Programming4us
         
 
 
Programming

Parallel Programming with Microsoft .Net : Parallel Aggregation - The Basics

12/9/2010 3:21:27 PM
The most familiar application of aggregation is calculating a sum. Here’s a sequential version.
double[] sequence = ...
double sum = 0.0d;
for (int i = 0; i < sequence.Length; i++)
{
sum += Normalize(sequence[i]);
}
return sum;

This is a typical sequential for loop. In this example and the ones that follow, Normalize is a user-provided method that transforms the input values in some way, such as converting them to an appropriate scale. The result is the sum of the transformed values.

The Microsoft® .NET Framework Language Integrated Query (LINQ) provides a very simple way to express this kind of aggregation. Languages such as C#, F#, and Microsoft Visual Basic® development system provide special syntax for LINQ. The following LINQ expression calculates a sum.

double[] sequence = ...
return (from x in sequence select Normalize(x)).Sum();

The LINQ expression is a sequential operation whose performance is comparable to the sequential for loop shown in previous example.

To convert a LINQ-to-Objects expression into a parallel query is extremely easy. The following code gives an example.

double[] sequence = ...
return (from x in sequence.AsParallel()
select Normalize(x)).Sum();

If you invoke the AsParallel extension method, you’re instructing the compiler to bind to PLINQ instead of to LINQ. The program will use the parallel versions of all further query operations within the expression. The Sum extension method executes the query and (behind the scenes and in parallel) calculates the sum of the selected, transformed values.

This example uses addition as the underlying aggregation operator, but there are many others. For example, PLINQ has built-in standard query operators that count the number of elements and calculate the average, maximum, or minimum. PLINQ also has operators that create and combine sets (duplicate elimination, union, intersection, and difference), transform sequences (concatenation, filtering, and partitioning) and group (projection). These standard query operators are sufficient for many types of aggregation tasks, and with PLINQ they all can efficiently use the hardware resources of a multicore computer.

If PLINQ’s standard query operators aren’t what you need, you can also use the Aggregate extension method to define your own aggregation operators. Here’s an example.

double[] sequence = ...
return (from x in sequence.AsParallel() select Normalize(x))
.Aggregate(1.0d, (y1, y2) => y1 * y2);

This code shows one of the overloaded versions of the Aggregate extension method. It applies a user-provided transformation to each element of the input sequence and then returns the mathematical product of the transformed values.

PLINQ is usually the recommended approach whenever you need to apply the parallel aggregation pattern to .NET applications. Its declarative nature makes it less prone to error than other approaches, and its performance on multicore computers is competitive with them. Implementing parallel aggregation with PLINQ doesn’t require adding locks in your code. Instead, all the synchronization occurs internally, within PLINQ.

If PLINQ doesn’t meet your needs or if you prefer a less declarative style of coding, you can also use Parallel.For or Parallel.ForEach to implement the parallel aggregation pattern. The Parallel.For and Parallel.ForEach methods require more complex code than PLINQ. For example, the Parallel.ForEach method requires your code to include synchronization primitives to implement parallel aggregation.

Other -----------------
- Developing an SEO-Friendly Website : Creating an Optimal Information Architecture (part 4)
- Developing an SEO-Friendly Website : Creating an Optimal Information Architecture (part 3)
- Developing an SEO-Friendly Website : Creating an Optimal Information Architecture (part 2)
- Developing an SEO-Friendly Website : Creating an Optimal Information Architecture (part 1)
- Cloud Security and Privacy : Governance, Risk, and Compliance (GRC)
- Cloud Security and Privacy : Internal Policy Compliance
- jQuery 1.3 : Improving a basic form (part 8) - Checkbox manipulation
- jQuery 1.3 : Improving a basic form (part 7)
- jQuery 1.3 : Improving a basic form (part 6)
- jQuery 1.3 : Improving a basic form (part 5) - Conditionally displayed fields
- jQuery 1.3 : Improving a basic form (part 4)
- jQuery 1.3 : Improving a basic form (part 3) - Required field messages
- jQuery 1.3 : Improving a basic form (part 1) - The legend
- jQuery 1.3 : Improving a basic form (part 1) - Progressively enhanced form styling
- Changes to Privacy Risk Management and Compliance in Relation to Cloud Computing
- Cloud Security and Privacy : What Are the Key Privacy Concerns in the Cloud?
- Cloud Security and Privacy : What Is the Data Life Cycle?
- Making Your Site Accessible to Search Engines
- Security Management in the Cloud - Security Vulnerability, Patch, and Configuration Management (part 2)
- Security Management in the Cloud - Security Vulnerability, Patch, and Configuration Management (part 1)
 
 
Most View
- SQL Server 2008: Administering Database Objects - Working with Tables (part 2) - Primary Key Constraints & Unique Constraints
- Microsoft Exchange Server 2003 : Administering Public Folders
- Windows Server 2008 : Configuring IIS Security (part 6) - Understanding Handler Mappings
- Microsoft SQL Server 2008 R2 : Setting Up Replication (part 2) - Creating a Publication
- Visual Studio Team System 2008 : Web test editor (part 3) - Toolbar properties
- Windows Server 2008 : Configuring Windows Media Services (part 9) - Using the Multicast Announcement Wizard
- Security in Cloud Computing (part 4) - Audit and Compliance
- Active Directory 2008 : Delegating the Support of Computers (part 1) - Understanding Restricted Groups Policies
- Exchange server 2010 : Troubleshooting Tools (part 1)
- Windows Server 2008 : Ongoing Backup and Recovery Preparedness
Top 10
- Implementing Edge Services for an Exchange Server 2007 Environment : Utilizing the Basic Sender and Recipient Connection Filters (part 3) - Configuring Recipient Filtering
- Implementing Edge Services for an Exchange Server 2007 Environment : Utilizing the Basic Sender and Recipient Connection Filters (part 2)
- Implementing Edge Services for an Exchange Server 2007 Environment : Utilizing the Basic Sender and Recipient Connection Filters (part 1)
- Implementing Edge Services for an Exchange Server 2007 Environment : Installing and Configuring the Edge Transport Server Components
- What's New in SharePoint 2013 (part 7) - BCS
- What's New in SharePoint 2013 (part 6) - SEARCH
- What's New in SharePoint 2013 (part 6) - WEB CONTENT MANAGEMENT
- What's New in SharePoint 2013 (part 5) - ENTERPRISE CONTENT MANAGEMENT
- What's New in SharePoint 2013 (part 4) - WORKFLOWS
- What's New in SharePoint 2013 (part 3) - REMOTE EVENTS