CRM Middleware Survival Guide (Relevant for Components CRM-MW*)
In the context of an SAP CRM landscape, Middleware refers to the R3 adapter, which is used to transfer data from the CRM to external systems such as R3, mobile client, Groupware and vice versa. The data are sent via qRFC (queued remote function calls). The Middleware is also used to transfer data internally from CRM Online to the CDB.
Types of Data Transfer/Loads using Middleware:
(i) Initial load (can be from ECC to CRM, CRM to ECC, ECC to CDB or CRM to CDB.).
(ii) Delta load (only in the direction R3 to CRM)
(iii) Upload (only from CRM to R3)
(iv) BDoc Notification (for delta changes in CRM)
(v) Request (uses the same framework as the initial load)
(vi) Synchronization (also known as a Compare Load. Similar to initial load but can only be used for customizing objects that update the CDB.)
(vii) DIMa load (Data Integrity Manger. Use to compare business object data between different databases. Please refer to note 531217 for further details.)
The R3 adapter is used for all BDoc transfer to and from the CRM. It is NOT used for transferring IDoc’s or in conjunction with the XIF adapter. For problems with the XIF adapter and IDoc transfer, please refer to notes and documentation corresponding to the appropriate application.
Databases on the CRM Server
A CRM system resides on a single physical database; however, from a logical perspective this can be viewed as consisting of two databases:
(i) CRM Online. As the name suggests, this database contains data relevant for online processing on the CRM system
(ii) CDB (Consolidation Database). This database contains data relevant for the mobile client. If the mobile scenario is not used, then this database need not be updated.
Types of Download Object Used in CRM:
There are 3 types of download objects:
Application Objects (Defined in transaction R3AC1) - These objects are used to transfer data used in normal, every day application processing, such as material data, business partners, sales documents etc. By default, after the initial load of an application object is performed, the delta download is activated. Afterwards, any changes made to corresponding data in the R3 are downloaded to the CRM. However, the delta load is relevant only for objects with flow defined from R3 to CRM and for data exchange in this direction.
Customizing Objects (Transaction R3AC3) – Customizing in an R3 SAP system is performed using the IMG (transaction SPRO). The IMG is also used for CRM customizing, but much of the customizing that governs how application data are configured must be synchronized between the CRM and R3 using customizing Adapter Objects.
Condition Objects (Tr R3AC5)
This transaction contains all Objects relevant for Conditions. These include Condition objects itself, which usually are generated at the customer site using a special report and condition customizing objects.
Important Transactions for Analysing MW Issues:
R3AS: Start Initial load
R3AC1: Adapter Object Overview (Application Objects)
R3AC3: Adapter Object Overview (Customizing Objects)
R3AC5: Adapter Object Overview (Condition Objects)
768503 Documentation for CRM Administration
765236 FAQ: Queueing in CRM and R/3
` Some Background Information on qRFC’s
The R3 Adapter uses queued Remote Function Calls (qRFC’s) to transfer data to and from the CRM system. The advantage of using qRFC’s is that the data are sent in sequence and if the one entry is stopped for any reason, then subsequent entries will not be processed until this entry has either been deleted or successfully processed. Thus, data consistency will be maintained, as long as a stopped entry is not manually deleted.
Important qRFC Transactions:
SMQ1 – Monitor Outbound Queue
SMQ2 – Monitor Inbound Queue
SMQR – Inbound Queue Scheduler
SMQS – Outbound Queue Scheduler
Refer to Note “765236 - Queueing in CRM and R/3” for details what queue names mean and their usage.
Note that queues in SMQ2 (ie inbound queues) must be registered in SMQR to be processed automatically. If the queue type has not been registered, then such queues will remain in status “Ready” until they are manually activated within SMQ2. It is possible to use a wildcard entry “*” for the queue name specified in SMQR.
The Outbound Queue Scheduler (SMQS) uses destination names rather than queue names. For outbound queues, automatic processing requires that the corresponding destination be registered in SMQS. (Exception: CRM_SITE_* queues do not have to be registered.)
Sequence of events during initial load from R3 to CRM:
(i) R3AI_ outbound queue is created in SMQ1 of the CRM. This requests data from the ERP Backend.
(ii) When the ERP system receives the request from the CRM, an R3AD_ entry is created on the R3 outbound queue. This entry contains no data; it is simply added to prevent any delta loads from taking place before initial loads finishes, thus ensuring data consistency. Therefore this IS NOT AN ERROR. During MATERIAL initial load a STOP entry is created to prevent all object downloads (IE R3AD*). Again, this is normal, and it required because of dependencies with other objects. The STOP entry will be removed once the initial load completed.
(iii) An R3AI_ outbound queue is created on the ERP system during the data selection.
(iv) When the data selection on the ERP side has finished, the data are imported into the CRM using queue RAI_. If filed USE_IN_Q in R3 table CRMRFCPAR is marked with an ‘X’ (this is the recommended setting), this will use inbound processing; if not, this will appear as an outbound queue.
Troubleshooting Problems with Loads from ERP to CRM and CRM to ERP
Check for Corresponding BDoc’s in SMW01. Refer to section
“BDoc Analysis” below.
For initial load, check each of the queues described above. For delta load, check R3AD* queues (on CRM and R3 side). For upload, check R3AU* queues.
.Queue naming conventions are described in detail in note 765236.
Corresponding queues in status SYFAIL
Double-click on the SYSFAIL and see what error is given. Perform note/message search on the error
If the outbound queue in the CRM is stuck in SYSFAIL, there will usually be a corresponding short dump. If the destination is R3, this dump will appear in the R3 backend. Reprocessing the queue will reproduce the dump.
Failed outbound queues on the R3 will also usually have a corresponding short dump.
If the inbound queue on the CRM has status SYSFAIL, check for corresponding BDoc’s in transaction SMW01. Please refer to the section “BDoc’s in Error/Intermediate Status”. Also, check for corresponding short dumps on the CRM.
Corresponding queues in status STOP
A STOP entry is set on an RFC queue by the relevant application. During data transfer from R3 to CRM, there are 3 common reasons for STOP entries:
The STOP entry added to the ERP outbound queue described in step (ii) of “Sequence of events during initial load from R3 to CRM” directly above. As mentioned previously, this is normal.
A STOP entry has been set y the application. In this case, search for a corresponding BDoc in SMW01 and see if there is a corresponding error.
The entry has been manually set by a user. In this case, it should e possible to start the queue again from within SMQ1 or SMQ2 by selecting the queue and clicking on Edit -> Unlock.
Queues in Status “Waitupda”
An update termination has occurred. The reason for the update error must be determined from the application side.
Queues Remain in Status “Ready”
Status “Ready” does not indicate an error; it indicates that the queue is ready for transmission. It may be that the queue is waiting for a dialog work process to become free, or that the queue has not been registered (please refer to section xxxx). However, if queues remain in this status permanently, or for a very long time, then this indicates a problem.
(i) First ensure that the queue is registered in SMQR (in the case of inbound queues), or that the destination is registered in SMQS (in the case of outbound queues).
(ii) There are insufficient free dialog work processes for the number of queues that must be processed. In this case, the most likely common causes are the ones described in notes 356228 and 763680. Check for these symptoms.
Further points to note:
After the implementation of note(s) 356228 and/or 763680, the speed with which new queues are created will decrease. However, it is still necessary to permit the queues that already exist to be processed in order to maintain data consistency. This may take some time, but due to the reduced rate of newly-created queues, the backlog should eventually clear and then, assuming the system has sufficient hardware, the bottleneck should disappear. Please note that we cannot give specific recommendations concerning how many queues the system should be able to handle; therefore, we cannot give specific recommendations with regard to the values that should be set for the parameters described in notes 356228 and 763680. This is a matter for the customer to discuss with his/her Basis Administrator.
Please note: CSA* queues should NEVER be deleted on a productive system. Such queues are created by the update task that occurs when data are saved on the CRM Online database. The information they contain is required in order to create corresponding outbound BDoc’s in the correct sequence. If CSA* queues are deleted, the sequence of processing of outbound BDoc’s is lost.
However, in the case of R3AD* queues, if the backlog is so large that the customer feels the entries must be deleted, there is an option to do this. In this case, the reports mentioned in note 757920 can be scheduled to run in the background. Afterwards, the data consistency could be ensured by either a) repeating an initial load or b) using the Data Integrity Manger tool (transaction sDIMA). For more details of the fucnionality of DIMA, please refer to note 531217. Using DIMA is more satisfactory as regards performance. However, before deciding to use DIMA the customer should refer to the relevant application note for the business object in question, in order to see if there are any limitations. Each relevant note is listed in note 531217.
Another possible reason for insufficient work processes is that the RFC parameters are configured badly. In such a case, there will most likely be some free dialog work processes on the system. However, the symptoms described in first paragraph of the Solution section of Note 527481 should be checked. If the behaviour described is observed, this note should be recommended, along with note 74141. In case of further questions from the customer, the message should be forwarded to component BC-MID-RFC.
(iii) If neither of the reasons described above is relevant, then this is almost certainly a BC-MID-RFC issue. The problem is with the queue scheduler itself and not the Middleware. However, it is worthwhile to check the following notes (for inbound queue only): 1067854, 909869.
No Queue or BDoc Errors but Not All Data Transferred
Data transferred that Do Not Correspond to Filter Settings
Check note 788822 - Filtering does not work.
Experience shows that this note is comprehensive with regards to filter problems. If a customer says there is still a problem with filtering after checking the note, you should check it on the R3 and CRM system yourself. A common issue that is often overlooked is entries missing from tables CRMMAPTAB and CRMINTBAP. If these entries do not exist, then filtering is not possible. Whether corresponding entries are permitted, and what exactly these entries should be, is something that can only be determined from the application side. The MW uses the entries if they are there, but if they are missing, then the problem should be investigated in the application component responsible for the download object. There may be a perfectly good reason why filtering is not allowed on certain tables or fields, or it could be that entries can be added to permit the filtering.
Check also below, section titled “Sales BDoc’s Look Successful in SMW01 But Data are Missing”
Changes Made on the CRM are not sent to the R3 System (ie Upload is not performed)
For the purposes of CRM Middleware, the term “upload” describes the transfer of changes made on the CRM system to the R3 backend. Such changes are sent in R3AU* queues, which created in SMQ1 (outbound processing) of the CRM. No queues relating to upload are created on the R3 Backend side. Data will only be sent if a corresponding subscription has been assigned to the R3 site in the Admin Console (transaction SMOEAC). The Middleware provides the framework for this, but which subscriptions are required for which data is the responsibility of the application.
Note: Initial load from CRM to ERP uses the normal R3AI* queue and is not covered by the term “upload” in this context. However, customers sometimes confuse the nomenclature. If a customer describes a problem with an upload, it is important to determine whether the issue relates to delta changes made on the CRM, as opposed to an initial load from CRM to R3.
If the relevant queue has been created, please check the status and analyse it as described in the section above, “Troubleshooting Problems with Loads from ERP to CRM and CRM to ERP”.
Filters Not Used During Upload (R3AU* Queues)
What criteria fields can be used is determined by the application, not the Middleware. How they should be defined is considered a matter for consulting rather than support.
(For Business Partners, please see the solution offered at the end of message 0187845 2007.)
Delta Load Doesn’t Work
Check Note 430980 - CRM Server: Analysis in delta data exchange R/3->CRM
A delta load is normally triggered automatically once an initial load has been performed. Then, an entry for the corresponding object class is added to transaction R3AC4 on the CRM. Such entries can also be added manually. The delta load is triggered by an event called on the ERP. This event must have a corresponding entry in R3 table TBE31. This table is normally updated by the CRM when the corresponding object class has been added in R3AC4. However, if the delta load is not triggered, then table TBE31 on the ERP backend should be checked. Some object classes require more than one entry, so if only some data belonging to a particular object to are loaded, TBE31 should also be checked. Which specific entries are required individual objects is a matter for the application, not the Middleware.
Normally it is easy to find the required entries by performing a CSS message and/or note search using the following search criteria:
“TBE31” AND (“” OR “”).
Refer to the document “Best Practice for BDoc Message Analysis“, attached to note 768503. This document is also available for customers.
Tr SMW01 can be used to check the status of all BDoc’s. The search can be filtered according to status, BDoc GUID, BDoc type and queue name. From 4.0 onwards, it is necessary to hit the third button in from the top left-hand corner in order to expand and see all of the entry fields.
Additional Points to Note:
BDoc’s in Error Status:
Errors can be seen by clicking on the red “Show BDoc msg/errors/receivers” button. Validation errors are always raised by the application; in most cases technical errors are also the responsibility of the application
In order to find the responsible component, click on the long text and look for the component responsible for the validation module mentioned (ie “service that caused the error”) and/or the error code given.
Sales BDoc’s Look Successful in SMW01 But Data are Missing
It is possible in some cases that a BDoc with a green light may have still have an error or errors. This can occur if a partial confirmation has been sent. In such cases, you can see any errors by clicking on the “Show BDoc msg errors/receivers” button, or else by checking the application log, transaction SLG1.
BDoc’s in Intermediate Status:
I02 – In addition to the reasons described in the Best Practice Guide, notes 356228 and/or 763680 may also be relevant. If these symptoms occur, please refer also to section titled “Queues Remain in Status ‘Ready’ - Further points to note”.
I04 – As stated in the Best Practice Guide, in order to resolve such issues it is necessary to “solve the problem with the open update task”. This requires application expertise; Middleware is not responsible for the update failure. Once the underlying reason for the update error has been resolved, the BDoc’s should be handled as described in corresponding section of the guide (p18-19).
Classic Data Empty
If a BDOc is created with no classic (header) data, then the problem should be investigated from the application side. Once the Middleware has created a BDoc, the application is responsible for filling the header data
Note 350176 - CRM/EBP: Performance improvement during exchange of Data Exchange
Delta load (R3AD* queues): Note356228 -qRFC occupies all work processes on the R/3 backend or CRM
CSA* Queues: Note 763680 - CSA* qRFC queues occupy all work processes on CRM
Ensure the latest version of the qRFC has been obtained as described in note 438015.
If so, ask the customer to check if basis check Note 742950 - Performance affected on Oracle DB with Supplement 11 is relevant.
Refer also to sections “Queue Analysis - Queues Remain in Status ‘Ready’” and “BDoc’s in Intermediate Status: I02” below, along with the section on BDoc’s in status I02 in the document “Best Practice for BDoc Message Analysis“, attached to note 768503
Note that it is sometimes difficult to determine for sure whether a performance problem is due to issues with MW, the application or basis. It is often helpful to check in SM51 to see if there are long-running sequential reads for the RFC user (ie the user defined for the relevant RFC destination in SM59).
Performance problems with Replication and Realignment Queues (Queue Demon, Transaction SMOHQUEUE)
Implement Parallel Processing as described in note 453882.
Note also that a performance issue may arise with the queue demon if too many parallel processes are configured. Potentially, the largest number of dialog work processes the Queue Demon may try to use simultaneously is the sum of the values entered for the RRQUEUE_PARALLEL parameters. If this sum is greater than the total number of dialog work processes configured for the system, then the Queue Demon could try to utilise a greater of dia WPs than the system has available, thus causing a bottleneck. Since Production systems usually have a lot of additional dialog processing activity, it is recommended as a general rule of thumb that the total number of WPs made available for the Queue Demon is not much greater than 50% of the system’s available dia WPs. Transaction SM51 can used to determine how many dia WPs have been configured for any system.
Check transaction SMQS. On the client in use, there should be an entry for SAPCRM_MW_RR_000. In order to optimise performance, this entry should be excluded as described in note 480543. In client 000 there must be entries both for
SAPCRM_MW_RR_000 and for SAPCRM_MW_RR_, where is the client in use. All of these entries should also be excluded.
Queue Demon in Status Hold or Starting
Either of these symptoms can be the result of performance issues. Check the section “Performance problems with Replication and Realignment Queues”.
For Queue Demon queues going into status “Hold”, check for related short dumps. By double-clicking on the “Hold” entry you can recreate the dump. Check also transaction SM58 for error messages.
Admin Console (Transaction SMOEAC)
Every destination that the CRM system sends data to must be defined in the Admin Console. This includes the CRM system itself, and – assuming the mobile scenario is used - also the CDB.
Which data are sent to individual external sites (ie – sites other than the CRM or CDB) is governed by which subscriptions are assigned.