How to Properly Integrate Automated Data Capture Hardware with Your ERP System
This post was contributed by Sergey Bazhenov, CEO and Co-Founder of Cleverence, a Zebra Independent Software Vendor (ISV) Partner.
###
In my last post, I told you that integrating automated data capture devices like barcode scanners, RFID readers and mobile computers into ERP systems isn’t as easy as “plug and play.” And that’s true. But it’s not because the devices are difficult to integrate. It’s because the ERP system was originally built – and has continuously been maintained by ERP system vendors – as a form-fill type system. It has never been designed to handle fast, continuous data inputs, at least not in its out-of-the-box state. That’s why some extra work is needed upfront to ensure your ERP system can handle all the valuable data your workers are capturing and transmitting via automated data capture devices.
So, let’s talk about what it takes to integrate automated data capture devices with your ERP system the right way so that neither your IT teams nor your front-line workers balk about how hard, slow, complicated, etc. it is to get data from barcode labels or RFID tags into the ERP system and maintain a single source of near-real time truth.
The Best Way to Set Up Automated Data Capture Streams to the ERP System
First things first, you must remember that ERP systems are designed to handle large volumes of data and concurrent users, but their performance can degrade with an excessive number of transactions from automated data capture, even via some enterprise-grade remote desktops. As such, a direct, real-time integration with barcode, RFID and mobile computer devices may not be feasible. Something should be put in between, like a gearbox (if you are into cars), to accommodate for that. It should either be purpose-built middleware, some custom code on the ERP side, or both.
Moreover, ERP systems primarily focus on document management as their purpose is to constantly process document transactions that mirror completed business activities. These document-centric workflows are vital for data consistency. Sometimes you can't even save an incomplete document in the ERP, so closing it will lose the entire job. However, automated data capture is designed to process raw operational data before all the tweaks are done. Therefore, it’s essential for a line of business operation to save an incomplete job for later and not to lose what’s already done. That’s why middleware is necessary.
Most advanced middleware will do that entire job and exchange the data with the ERP system based only on fully qualified documents, not merely small transactions.
Therefore, this is how I do things (and you should too) in a receiving use case:
Get a purchase order from the ERP,
Do the full receiving with barcodes with middleware,
Fix all the issues, and
Return the resulting receiving document back to the ERP.
In setting up this data capture/input structure, I always give consideration to the client-side user interface (UI), specifically, the relationship between the automatically captured data and the business rules. You should too. UI design can dramatically improve data accuracy and operational efficiency. Middleware with its own client native application can squeeze this last bit of efficiency into your company’s pocket.
Now, I know you may also try – and struggle – to use remote desktop apps or browsers for automated data capture, but they are designed for general use and lack the precision needed for specialized automated data capture tasks. While they offer the convenience of accessibility from various devices, they often lack the fine-tuned optimizations necessary for automated data capture hardware. The latency issues in remote desktops, coupled with non-responsive designs in general browsers, can hinder rapid data entry and retrieval.
For instance, when you do a cycle count in a warehouse or in a store, you quickly stop looking at the screen of the mobile device because the task requires you to look at the products and barcodes. In a browser or a remote desktop, this will lead to lost scans. In the case of discrepancies, the software will try to attract your attention with some remote desktop screen or a web popup, but that will completely miss your attention because you are not looking at the screen. It might even try to play some sound. But what if all sounds on mobile devices at work are turned off? You will continue to scan, and you will lose all those scans.
Therefore, it is almost always recommended that you use a purpose-built native app that is integrated with the device’s firmware. Why? Well, the app can turn off the scanner on those occasions I just noted, so you will be unable to scan and will be forced to look at what happened on the screen. This will prevent errors while keeping the scanning fast and agile.
That is the proper way to integrate the automated data capture hardware with the ERP – “the best money can buy” (at least from my point of view). Now let’s talk about why it’s the proper way by looking at what happens if you try a different approach.
Common Mistakes to Avoid
As with any technology integration, there are always multiple approaches. Some are better than others, and it’s often hard to know which is best. Though your specific situation will warrant a very personalized recommendation, I can tell you what not to do in any situation if you want to remove friction, save time and money, and see the business improvements results promised by automated data capture technology. So, let me call out a few things that can get you into trouble and advise you on how to avoid them.
Mistake #1: Being Overly Reliant on ERP Vendors
One of the challenges business leaders and IT teams face when integrating automated data capture devices with ERP systems is the heavy reliance on ERP vendors to solve integration challenges. However, given that device integration is a highly specialized requirement, many ERP vendors may not fully understand or support it. And if they do, it will be very expensive.
To put it bluntly, most ERP systems were never designed with automated data capture/intake in mind. Their user interfaces were built for manual inputs. The speed, efficiency, and protocols of barcode scanners, RFID readers and other automated data capture devices can often cause problems in ERP systems. So, you must find a way to work around these design limitations, and leaning solely on ERP vendors may not be the best way.
Basically, you have two options:
Use only vendor-supported tools. You can opt to work with the ERP vendor and hope they have the specialized expertise to facilitate a smooth hardware-ERP integration and provide great support after the fact. However, there is a good chance they won’t since ERP vendors are not at the cutting edge of automation technology nor equipped with decades of automated data capture experience. The integration support they can provide is only for already outdated hardware, old standards and very basic processes – all the things that ERP systems are often known to be. And while some ERP vendors might initially provide integration support for a couple of specific automated data capture devices, support with maintaining and updating these devices and integrations can often fall by the wayside. As a result, you can't solely rely on ERP vendors for the ongoing success of your automated data capture integrations (even if they are able to help you through the actual integrations). Of course, you can’t just abandon your ERP system given how central it has probably become to many business functions. That means you need to strongly consider your second option.
Use third-party middleware or make your own. This enables you to gain and maintain full control over the parameters of the ERP system and the latest technology that may be connected to the system. On the negative side, there are bugs, development costs and often delays that you’ll have to accept and be prepared to manage.
My recommendation: Start by spending time trying to get support from the ERP vendor for what you want to achieve and, if they can’t give you what you need (without compromise), you go for the second option.
My team and I recently worked with a customer on a case where an ERP vendor provided no application programming interface (API), prohibited direct database access (because the ERP exclusively locks the access), and asked for $70K to complete the mobile devices’ integration. This was not a good deal for the customer by any means, so they called us. We were able to utilize the "Import from Excel" \ "Export to Excel" functionality of that ERP for stock reports and some documents to make it look that there is a real-time data exchange between the ERP and mobile devices with a native app to facilitate warehouse operations.
Mistake #2: Failing to Make Case for Middleware
Middleware is basically a piece of software that ties together two other pieces of software or hardware. It can be really simple or really complicated. A printer driver is middleware, as is a web server like Apache. Browsers, remote desktop tools and fully-fledged client-server mobile apps with web service are also examples of middleware that you may rely on at some point.
In the case of automated data capture integrations, all that middleware will probably require some licensing and implementation efforts. That’s why I don’t recommend you use consumer-grade remote desktop apps or browsers. They might seem like a simple solution, but they often fall short when it comes to supporting barcode scanners, RFID readers and the specialized functions of other automated data capture hardware. The consumer browser will lose some scans, the consumer remote desktop will cut long trails of 0s in a barcode, and neither is capable of the thousands of keyboard types per second that may stem from an RFID reader.
For instance, a barcode scanner can behave simply as a keyboard and may be okay for simple short barcodes. But if you need to scan thousands of logistic labels with several GS1 barcodes, and you don’t want a separate warehouse management system (WMS) for that, then that’s a job for purpose-built middleware or a custom ERP screen.
In other words, you will likely require a native app or enterprise-grade versions of browsers and remote desktops that are designed to interact efficiently with automated data capture devices.
Now, let’s examine the pros and cons of these kinds of middleware.
Remote Desktop: Primarily for legacy systems, a remote desktop provides access to the ERP system from a device equipped with a barcode scanner or RFID reader. An enterprise-grade remote desktop app will be needed to ensure the ERP system can properly interface with the automated data capture hardware.
Web Apps: While web apps can provide a better user experience, they will also require an enterprise-grade browser to really support automated data capture hardware. The use of a special browser will limit the ability to leverage the latest web frameworks and JavaScript features. The compatibility and feature-support of the chosen enterprise browser will be a critical factor to consider.
Native Apps: A native app can be designed from the ground up to support automated data capture hardware and provide an optimal user experience. Native apps provide superior performance and an ideal user experience, exploiting the device's full capabilities.
While native apps require significant time and resources to develop and maintain, they offer the best functionality. A well-designed UI can provide step-by-step guidance, decision-making support, error prevention and correction mechanisms, and real-time feedback. And it will become better and cheaper over time.
If you have a web-based ERP, a web app offers a straightforward solution. However, it depends heavily on a reliable internet connection (so you’ll need to ensure you have a reliable wireless network in place), and it may not perform optimally compared to a native app.
The remote desktop option is slightly worse. While an effective stopgap solution, it may not offer an optimal user experience. Although, an internal developer with enough effort can make custom screens for specific device resolution that will fit the purpose (in case you are interested in this option).
Mistake #3: Not Considering the Whole Tech Stack and Integrated System
Your company’s tech stack is much like the machinery of a well-oiled machine; every component has its function, and when chosen carefully, the result is a harmonious system. When you decide on each component of your system, factors like functionality, manageability, and interoperability should be at the forefront. From a hardware and software perspective, ensuring compatibility and efficient communication is vital.
The stack is basically a range of operating systems (OS) and its versions (both mobile and desktop), the ERP system and its versions, and other systems. Simple enough, right? Well, it depends whether things are cloud or on-premise and what types of developing environments with which your developers have experience.
There are three integration methods worth considering when you need to make automated data capture devices and middleware work with your ERP of choice. Things can get very technical pretty quickly here, but I’m going to try to keep them high level so you can start to see the simplicity or complexity of different options:
UI-Based integration uses tools built for humans, i.e., existing screens or buttons like “Save as Excel…”\''Populate from Excel…” to exchange data collected by barcode scanners and RFID readers. This is the best case for remote desktop connection or file exchange. There is also a thing called Robotic Process Automation (RPA) where bodiless ethereal robots move the mouse and type on a keyboard to mimic what people do, just 100 times faster, retyping automatically captured data into the ERP.
API-Based integration leverages the ERP system's APIs to facilitate data transfer. This method is generally safer than direct database integration, as it uses the ERP system’s own mechanisms for data access and modification. However, it requires robust APIs from the ERP vendor and can be limited by the API's capabilities.
Direct Database integration involves connecting the automated data capture system directly to the ERP system’s database. This method offers simplicity and direct control over data flow but can be risky if not handled correctly. You could corrupt your data and should avoid that, so this method is not good. (Funny thing is that when some API is lacking, the way you implement a new API is with direct database access.)
Now, remember that the tech stack vastly influences your range of motion. Sometimes there is no API, no database access, no developers, and only ERP screens that don’t know what a barcode is. In such cases, a human or RPA platform will be retyping the data received via email from a WMS or other system.
Which option is right for you?
You can hire third-party consultants or services that specialize in ERP and automated data capture device integration to help you figure out the best approach, which I always advise when you’re trying to use the ERP system as your single source of truth. These firms can provide expert advice and support, making them an excellent option if you lack the resources to develop in-house expertise.
Alternatively, you can develop in-house expertise in automated data capture integration, be it through training existing staff or hiring new employees with the required skills. This ensures that your business can independently maintain and adapt your device-ERP integration as your needs evolve and as new devices come onto the market. However, even if you take this route, it never hurts to have an outside expert on standby to assist as needed. You can always benefit from an outside perspective.
In Other Words
The promise of automated data capture is alluring – it’s a quicker, more accurate way to keep your systems and people up to date on what is happening across your operation. It’s the key to real-time visibility! But the real magic happens when ERP systems and automated data capture devices synchronize harmoniously. You should take the time to learn about some of your middleware options and investigate their integration and performance capabilities yourself. Don’t just rely on what your ERP vendor says. They are neither automated data capture nor hardware integration specialists, and they are not well-versed on how to configure technology systems to become productivity-supporting tools.
Plus, the middleware designed to facilitate these types of integrations comes in all flavors, with different client-side experiences and capabilities intended to solve a variety of problems. Each of these client-side solutions comes with its own challenges, though, requiring specific enterprise-grade apps or browsers to support automated data capture hardware correctly.
So, please don’t try to figure this out on your own – or with your ERP vendor only. Talk to someone who has been doing it for many years – someone who has already experienced, and solved for, common pain points in the integration process. Listen to what they recommend, as they truly have your best interest in mind. They don’t want you to experience what others have, trying to force fit devices into an ERP system that, on its own, isn’t meant to support the type of information flows required to keep up in this high-speed, on-demand economy. They want to help you make the most of what you have (the ERP) so that your team can make the most of their time (with automated data capture).
###
Editor’s Note:
If your ERP is struggling to intake the data being automatically captured by the barcode scanners, RFID readers or mobile computers your team is using, or you know that you’ll need to get these devices in sync with your ERP system soon, you can send your questions to Sergey.
###
Additional Insights You May Find Helpful:
