How Machine Learning Can Transform Digital Asset Management

As the use and need for digital assets increase, so too does the cost and complexity of Digital asset management (DAM) — especially in a world where people are adopting devices with screens of all sizes (e.g., desktop, mobile, tablet, etc.).

DAM, however, is a challenge for many organizations. It still involves frequent manual labor, but machine learning is starting to change that.

Machine learning has already given us self-driving cars, speech recognition, effective web searches, and many other benefits over the past decade. But the technology can also play a role in classifying, categorizing, and managing assets in the years to come.

Machine learning can support DAM in areas such as face recognition, image classification, text detection, people recognition, and color analysis, among others. Google PlaNet, for example, can figure out where a photo was taken based on details embedded in it. Google Photos is using it to improve the search experience. Machine learning has already taken a role in image spam detection. Taken together, this all points to the need for DAM tools to start incorporating advanced machine-learning capabilities.

A Practical Test

Recently, Google released its Cloud Vision API. The Google Cloud Vision API enables developers to understand the content of an image by encapsulating powerful machine-learning models in an easy-to-use REST API. It quickly classifies images into thousands of categories (e.g., “sailboat”, “lion”, “Eiffel Tower”, etc.). It detects individual objects and faces within imagines. And it finds and reads printed words contained within images.

For Razorfish, this was a good reason to explore using the Vision API together with a DAM solution, Adobe AEM DAM. The result of the integration can be found on github.

Results screenshot

We leveraged text-detection capabilities, automation classification techniques, and the landmark detection functionality within Google’s API to automatically tag and assign other metadata to assets.

Benefits and Setbacks

Integrating the Vision API provided immediate benefits:

  • Automated text detection can help in extracting text from images, making them easily accessible through search.
  • Automated landmark detection helps in ensuring that the appropriate tags are set on digital assets.
  • Auto-classification can support browse scenarios for finding the right assets.

But there were also some shortcomings. For example, an image of a businesswoman in a white dress was identified as a bride. In other instances, the labels were vague or irrelevant. Though inconvenient, we expect these shortcomings to improve over time as the API improves.

Even with these drawbacks unaddressed, automated detection is still very valuable — particularly in a DAM scenario. Assigning metadata and tags to assets is usually a challenge, and automated tagging can address that. And since tags are used primarily in the authoring environment, false classifications can be manually ignored while appropriate classifications can help surface assets much broader.

The Evolution of DAM Systems

One frequent point in implementing DAM systems is asset migration. I have seen many clients with gigabytes of assets wonder whether to go through the tremendous effort of manually assigning metadata to them.

There’s a quick fix: Auto-classification techniques using machine learning will improve and speed up this process tremendously.

With the benefits around management and migration, machine learning and other intelligence tools will therefore start becoming a key component of DAM systems — similar to how machine learning is already impacting other areas.

Lastly, incorporating machine learning capabilities in DAM solutions will also have architectural implications. Machine intelligence functionality often uses a services-based architecture (similar to the APIs provided by Google) as it requires a significant or complex set of compute resources. As DAM systems start to incorporate them at its core, it will be more difficult for those solutions to support a classic on-premises approach — causing more and more solutions to migrate to a hosted software as-a-service (SaaS) model.

Bottom line? Consider incorporating machine learning into your DAM strategy now, and look at how it can be applied to your digital asset management process.

written by: Martin Jacobs (GVP, Technology)

Diffusing Automation Arguments: The Inevitability of Automation

As mentioned in one of my previous posts, delivering a successful Cloud architecture necessitates the use of automation. Unfortunately, replacing manual tasks with code takes effort, and is therefore not always used. Here are some key arguments against the adoption of automation:

Priority

“We are already on a tight deadline with all the application features that need to be incorporated.”

Automation is critical to the success and longevity of your product. What’s also true, though, is that this is an industry of tight deadlines, stretch goals, and additional features. You might wonder if you have time to automate.

In this case, unit testing is an interesting comparable situation. Often times, unit testing hasn’t always taken priority in the application development process due to time constraints. It has been put off until the end of development phase with a secondary status. However, unit testing has slowly received the priority it deserves, as it has become clear it provides the benefits in the long run.

And as much as testing is important, automation is even more critical. Automation is an actual part of your runtime application, and should be treated at the same level as your code. The features and capabilities for automation should therefore be included in the application/solution backlog and should be given the same treatment as other features and functionality.

Skills

“We don’t have the skills in-house. Even if we were to use a vendor, we wouldn’t be able to maintain it.”

No doubt, automation is a serious challenge. Automation requires a fundamental shift in mindset for organizations around the need to develop these skills. You may remember that in the early days of web development, it took quite some time for front-end development to become a respected and critical role as say database administration. The automation architect will face a similarly arduous battle for the coming years. For any organization that leverages the Cloud and maintains their own technology platforms, it is a critical role that must be filled or grown within the organization.

Time

“It is faster to do it without automation.”

This is often true for the initial setup. However, considering how quickly Cloud architecture continues to evolve, the time gained from a hasty initial setup could quickly be lost in subsequent change management.

With Cloud architectures incorporating more distinct elements, ensuring consistency across environments is virtually impossible without automation. As a result, without automation, the likelihood of generating defects due to environment mismatches increases quickly when your Cloud architecture grows.

Technologies in Use

“The application technologies we use don’t support automation”

As you architect your application, you identify critical non-functional requirements. For example, security and performance are always part of the decision criteria for the overall architecture stack, and if the technologies selected cannot support the level of performance required, you would evaluate alternative options and select and migrate your architecture to the new solution.

The same applies for automation. If automation cannot be supported with the existing technologies, it is necessary to look at alternatives, and evolve your architecture.

Overwhelming Choices

“We are confused by the technology landscape.”

The amount of solutions in the marketplace can certainly feel paralyzing. There’s Ansible, Chef, and PuppetLabs. There are provisioning tools such as AWS Cloud Formation, Heat, Terraform, and Cloudify. Solutions are constantly evolving, and new vendors are always showing up.

It is difficult to make the right choice of technologies. The selection should be made with the same mindset as selecting the enterprise set of programming languages. It requires an evaluation of which is best suited for the organization. Additionally, a combination of these technologies might be the right solution as well. As you embark on applying automation, here are some tips for being successful:

  • Select a set of automation technologies and stick with it. There will always be pressure to explore alternatives, especially with a quickly changing vendor landscape, but it is important to fully understand your selected technologies before looking at alternatives.
  • Start simple. Amazon Elastic Beanstalk or Heroku are great ways to begin to incorporate automation into your application development workflow and understand how it can further drive productivity and quality.
  • Avoid the framework syndrome and focus primarily on building the automation that is needed for your application. Don’t try to build a framework for automation in the enterprise. The landscape is constantly evolving and frameworks quickly become outdated and superseded.

written by: Martin Jacobs (GVP, Technology)