Microsoft Dynamics AX 2012 Data Import Export Framework Web Services
Purpose: The purpose of this document is to illustrate how can leverage Microsoft Dynamics AX 2012 Data Import Export Framework Web Services to automate data import and export scenarios.
Challenge: Microsoft Dynamics AX 2012 Data Import Export Framework is a recommended solution for data import and various integration scenarios. Data import using Data Import Export Framework consists of 2 steps: copy data from source to staging table, copy data from staging table to target. These 2 steps can be executed as 1 step. Data Import Export Framework jobs can be set up for postponed execution in batch if needed. Some of integration scenarios require a high level of automation especially when you need to invoke Data Import Export Framework jobs from external application.
Solution: For data import and integration scenarios requiring a high level of automation you can leverage Data Import Export Framework Web Services. Microsoft Dynamics AX 2012 Data Import Export Framework ships with a number of Web Services which can be used for inquiries and invoking jobs.
Walkthrough
In this walkthrough I’m going to invoke Data Import Export Framework Web Services to automatically execute “Copy from source to staging” and “Copy from staging to target” jobs from external application. For the sake of simplicity I’ll create C#.NET Console application to do so.
Microsoft Dynamics AX 2012 Data Import Export Framework ships with the following Web Services:
<![if !supportLists]>- <![endif]>DMFDefinitionGroupService (create)
<![if !supportLists]>- <![endif]>DMFEntityWriterService (targetService)
<![if !supportLists]>- <![endif]>DMFExeStatusService (find, read)
<![if !supportLists]>- <![endif]>DMFProcessGrpService (find, read)
<![if !supportLists]>- <![endif]>DMFStagingService (checkConnection, exportToFile, write)
<![if !supportLists]>- <![endif]>DMFStagingWriterService (stagingService)
DMFServices
Let’s take a closer look at DMF Web Services
DMFDefinitionGroupService
DMFDefinitionGroupService (Class): Creates the new mapping as per current sample file.
Create (Operation): This method is used to create a record in a defination group and generates a mapping between source and staging. Source may be a file,odbc or xml. By using this service from outside of AX, a user can create a single processing group with multiple entities or multi-processing group with multiple entities.
DMFEntityWriterService
DMFEntityWriterService (Class): The DmfEntityWriterService class is used to import data from external file to AX.
targetService (Operation): Service entry point to import data from a file to ax.This method will check for the processing group, if exists then updates the record set in DMFDefinitionGroupExecution table. For each entity, mapping will be created between source to staging and staging to target. Once the mapping is created successfully, it will copy the data first from source to staging, validates the data and then copies from staging to target.
DMFExeStatusService
DMFExeStatusService (Class): The DMFExeStatusService class serves as the document service for the DMFExeStatus document type.
find (Operation): Finds data objects.
read (Operation): Reads a DMFExeStatus document that contains one or more data objects.
DMFProcessGrpService
DMFProcessGrpService (Class): The DMFProcessGrpService class serves as the document service for the DMFProcessGrp document type.
find (Operation): Finds data objects.
read (Operation): Reads a DMFProcessGrp document that contains one or more data objects.
DMFStagingService
DMFStagingWriterService (Class): Creates the Staging Writer Service class.
checkConnection (Operation): Checks the writer connection.
exportToFile (Operation): Service entry point to export data from staging to file.
write (Operation): Writes in Staging.
Please note that write method calls executeService method listed below.
executeService (Operation): To import data from a file to staging.This method will check for the processing group, if exists then updates the record set in DMFDefinitionGroupExecution table. For each entity, mapping will be created between source to staging. Once the mapping is created successfully, it will copy the data from source to staging.
DMFStagingWriterService
DMFStagingWriter (Class): Creates the Staging Writer class.
stagingService (Operation): Runs the staging service after populating the parameters.
As I mentioned before data import using Data Import Export Framework consists of 2 steps: copy data from source to staging table, copy data from staging table to target. Please find a detailed DMF User Guide here: http://technet.microsoft.com/en-us/library/jj225591.aspx
For my scenario I’ll leverage DMFStagingWriterService and DMFEntityWriterService DMF Web Services to copy data from source to staging and copy data from staging to target correspondingly.
First off I’ll deploy Inbound port which exposes DMFStagingService and DMFEntityWriterService DMF Web Services operations
Inbound port
For simplicity I’ll import Vendor groups data using a file
Given
I’ll start setting up Data Import Export Framework by creating a processing group
Processing group
And then adding Vendor groups standard template to it
Select entities for processing group
Next step will be to create a simple C#.NET Console application and add a reference to DMF Web Services to it
C#.NET Client application: >Staging
When Service reference is added I can invoke write operation of DMFStagingService DMF Web Service in C# as depicted below
Source code
using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using DMF.DMFServiceReference; namespace DMF { class Program { static void Main(string[] args) { DMFStagingServiceClient client = new DMFStagingServiceClient(); CallContext context = new CallContext(); context.Company = "usmf"; DMFStagingWriterContract contract = new DMFStagingWriterContract(); contract.parmDefinationGroupName = "Alex"; contract.parmExecutionID = "Alex-1"; contract.parmEntityNameList = "Vendor groups"; contract.parmFilePath = "C:\Users\Administrator\Desktop\Vendor groups.txt"; try { client.write(context, contract); } catch { } } } } |
Once copy data from source to staging is done we can review Execution history
Execution history
As the result the data will now be in appropriate Staging table
Result
At this point we can also write a code to import data from staging to target using C#.NET
C#.NET Client application: > Target
This time I’ll need to invoke targetService operation of DMFEntityWriterService DMF Web Service in C# as depicted below
Source code
using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using DMF.DMFServiceReference; namespace DMF { class Program { static void Main(string[] args) { DMFEntityWriterServiceClient client = new DMFEntityWriterServiceClient(); CallContext context = new CallContext(); context.Company = "usmf"; DMFEntityWriterContract contract = new DMFEntityWriterContract(); contract.parmDefinationGroupName = "Alex"; contract.parmExecutionID = "Alex-1"; try { client.targetService(context, contract, "Vendor groups", "1"); } catch { } } } } |
We can now review the result of the execution in Execution history
Execution history
Please note that when moving the data from staging to target the system will create a batch job. This can come handy in case you execute operations asynchronously (for example, targetServiceAsync operation) and some time is required to complete copy data from source to staging job and before copy data from staging to target job kicks off.
Batch job
When batch job is done you can also review infolog details as depicted below
Infolog
As the result a new Vendor group record gets successfully created
Result
For full scenario automation 2 code snippets provided above for copy data from source to staging and copy data from staging to target can be unified into a single method which call DMF Web Services operations synchronously or asynchronously
Similar to how I invoked DMF Web Services operations in C#.NET I could also do this from within X++
Please see examples below
X++ Client: > Staging
static void DMFStagingJob(Args _args) { DMFStagingWriterService service = new DMFStagingWriterService(); DMFStagingWriterContract contract = new DMFStagingWriterContract(); contract.parmDefinationGroupName("Alex"); contract.parmExecutionID("Alex-1"); contract.parmEntityNameList("Vendor groups"); contract.parmFilePath("C:\Users\Administrator\Desktop\Vendor groups.txt"); try { ttsBegin; service.write(contract); ttsCommit; info("Done!"); } catch { } } |
Result
X++ Client: >Target
static void DMFTargetJob(Args _args) { DMFEntityWriterService service = new DMFEntityWriterService(); DMFEntityWriterContract contract = new DMFEntityWriterContract(); contract.parmDefinationGroupName("Alex"); contract.parmExecutionID("Alex-1"); try { ttsBegin; service.targetService(contract, "Vendor groups", "1"); ttsCommit; info("Done!"); } catch { } } |
Result
Please note that there’re other examples of modern integrations with Microsoft Dynamics AX 2012 using DMF Web Services. For example, for MDM scenarios Microsoft Dynamics AX 2012 R3 is integrated with MDS using DMF Web Services, for data import scenarios you can also leverage the power of Data Import Export Framework in conjunction with RapidStart Services. Please find more details about MDS integration with Microsoft Dynamics AX 2012 R3 here: http://blogs.msdn.com/b/axsa/archive/2014/05/21/mdm-adapter.aspx
Summary: This document describes how you can leverage Microsoft Dynamics AX 2012 Data Import Export Framework Web Services to automate data import and export scenarios. This can come particularly handy in case you need to automatically execute DMF jobs from external application. We discussed what DMF Web Services are included with Microsoft Dynamics AX 2012 and how to invoke them in C#.NET and from within X++.
Tags: Microsoft Dynamics AX 2012, Data Import Export Framework, DIXF, Data Migration Framework, DMF, Web Services, Source, Staging, Target, Data Import, Data Migration, Automation.
Note: This document is intended for information purposes only, presented as it is with no warranties from the author. This document may be updated with more content to better outline the issues and describe the solutions.
Author: Alex Anikiev, PhD, MCP