How Machine Learning Can Transform Digital Asset Management - GAN

In previous articles I discussed the opportunities for machine learning in digital asset management (DAM). Most of these articles focused on leveraging machine intelligence to gain insights into asset content, and use that insight into the digital asset management, publishing and analytics processes. Examples that were highlighted were the automatic extraction of metadata from an image and detecting the location of a face to perform better cropping.

A different area is the concept of using machine learning to generate or extend digital assets. A recent article in the New York Times touched on this. The unique challenge around using machine learning for this purpose is that it is hard to actually test whether the machine learning algorithm has done a good job.

For example, it is relatively easy to verify whether the discovered metadata of an image is correct, and you could easily create a training set with labeled responses for the network. For a generated image, it is harder to determine whether this image is as good as a real image. Often times, there is not just one good answer. In many cases, there are many or even infinite options that are good.

To address this, the concept of generative adversarial networks (GAN) was developed. A GAN consist of two machine learning algorithms (neural networks), one to generate an image, and another assessing whether the image is real or generated. The 2 algorithms compete with each other. So instead of getting a training set to improve and learn, one network is tested against another network, allowing for unsupervised learning.

Generative Adversarial Network (GANs)

GANs were first introduced by Ian Goodfellow et al. in a paper published in 2014. The publication of this paper generated a lot of momentum, and we have seen some great ideas in the use of GAN’s for digital assets.

Some interesting examples are:

  • Generate high resolution images based on lowres. Use machine learning to generate a high resolution image from a lower resolution version. example result An example code repository:

  • Transform images. Automatically transform an image into a related image. For example, transform a horse into a zebra, a summer landscape into a winter landscape. example result An example code repository:

  • Transform styles. - Transform one image into another based on a provided style image. example result An example code repository:

  • Generate images. Automatically create real looking images. The generation of “celebrity” images was one of the highlighted areas in the referenced New York Times article, and the video below highlights some of output from that work.

    An example code repository:

  • Generate video frames. Use still photos to generate a video, or extend a video to generate additional video frames based on prior content. The image below from a Facebook post outlines the difference between the results computed from a traditional neural network vs a GAN (the adversarial GDL network) example result An example code repository:

  • Improved image manipulation. Support the image editing process to make all edits look more natural.

    An example code repository:

  • Generate images based on text. Automatically generate an image based on the provided textual description. A good summary of the techniques used for this are in this Microsoft post example result An example code repository:

GAN & Digital Asset Management

The examples above show some interesting new possibilities. Many of these will have a benefit in a DAM context. Obviously, improved asset editing tools built into the DAM will be a first step. The digital asset management system can also start incorporating a cognitive ability to interpret more abstract commands (e.g. ‘make this person look happy’) to enhance assets.

Going beyond the manual tools, the DAM itself might get an ability to automatically optimize and improve photography leveraging machine learning.

A frequent challenge for many clients is to scale asset production. When assets can be generated on demand, an ability to generate (related) images might become a built in function in digital asset management systems. This will open up new opportunities where asset production is currently a barrier. Richer personalized experiences online experiences, with unique image variations for each individual, can become a possibility.

Another interesting aspect could become digital rights management for assets. If an original photo is used for feeding the neural network, does its rights apply to the output as well?

Online shopping sites will be able to further enhance their ability to create rich immersive imagery and video to highlight products. Again, this can be connected to the visitor, creating a more individualized product catalog.

With the ability to generate unique natural looking variations of images, they can become their own “natural” QR code, providing another way to identify and track a visitor.

In the end, the concept of stock photos might evolve over time as well. With these new capabilities, high quality stock photos can potentially be generated, allowing us to move away from these overused standard photos.

written by: Martin Jacobs (VP, Technology)

How Machine Learning Can Transform Content Management- Part III

In a previous post, I highlighted how machine learning can be applied to content management. In that article, I described how content analysis could identify social, emotional and linguistic tone of an article. This informs a content author further on his writing style, and helps the author understand which style resonates most with his readers.

A follow up article outlined the use of Amazon’s Artificial Intelligence Services to perform text to speech translations.

At the most recent AWS re:Invent conference, Amazon announced Comprehend, a natural language processing (NLP) service that uses machine learning to find insights and relationships in text. This seemed like a good occasion to revisit the initial article, and explore how Comprehend can provide content intelligence.

Amazon Comprehend

Amazon Comprehend has two sets of services, one targeted for interactive use with responses available in hundreds of milliseconds, the other more batch oriented.

The interactive services has four core API’s:

  • Entity returns the named entities (“Person”, “Organization”, “Locations”, etc.) within the text being analyzed.
  • Key Phrases returns key phrases and a confidence score to support that this is a key phrase.
  • Sentiment returns the overall sentiment of a document (Positive, Negative, Neutral, or Mixed).
  • Language returns the dominant language in the text and a confidence score to support that a language is dominant.

The batch-oriented services can scan a set of documents, and return a summary of key topics in these documents.

Integrating Comprehend with Adobe AEM Content Fragments

Similar to the original article, we integrated AWS Comprehend with AEM Content Fragments.

The same piece of content was leveraged. Unfortunately, AWS Comprehend only supports up to 5000 characters, so a subset of the overall text was selected.

Entities, sentiment and language were assigned as metadata tags to the document. Only entities with more than two occurrences were included. Entities provide a categorization (such as date, location, event, organization, person, commercial_item, etc), and the category is included in the tag structure.

In this example, sentiment is also included. Overall, it would make sense to only include tone if a certain confidence threshold has been met.

The key phrase response was used to generate a highlighted version of the content fragment. Similar to sentiment and entities, each phrase is scored with a level of confidence, and this can be used to fine tune phrase detection.

If you want to explore the code, it is posted on Github as part of the repository.


In the original article, I listed a couple of potential reasons why leveraging machine learning in the content management process could be relevant:

  • How does content come across to readers? Which tones resonate the most? What writer is successful with which tone?
  • What topics are covered most frequently? How much duplication exists?
  • What is the article about?
  • What are the key topics covered in this article?

The Amazon Comprehend solution can help with some of these as shown in this proof of concept.

For topic classification, the approach taken here with entities can be valuable. Especially the ability to provide categorizations of entities can help with providing more easily readable insights.

Key phrase detection can help with quickly reviewing what an article is about, identify repetition and can especially be helpful in content management scenarios with high volume content production.

written by: Martin Jacobs (VP, Technology)

Top 5 highlights from Amazon’s re:Invent 2017 conference

Amazon Web Services’ annual re:Invent conference has wrapped up, and as usual, it was a great event. With an estimated 40,000 attendees, the event became again a bit larger than last year. I will highlight my personal top 5 in this article. The full list of all announcements can be found at

#5 AWS Fargate

Amazon had a number of container announcements, AWS Fargate being the most interesting. With AWS Fargate, there is no longer a need to provision, configure, and scale clusters of virtual machines to run containers, all of that is taken care of. To run an application, the only thing needed is a container image, CPU and memory requirements, networking and IAM policies.

#4 Video and Media Services.

The number of new announcements around video stood out. AWS DeepLens (a deep learning enabled wireless video camera) captured a lot of interest, but from my perspective, the most important one was the set of AWS Elemental Media Services to process, store, and monetize video.

These services are important step towards breaking up the various capabilities need for video management into individual pieces rather than one big service, allowing users to use what they need and merge with any of their in-house or competitive technologies if needed. The inclusion of Video on Demand and Live video as part of this set of services is terrific.

Combined with new Machine Learning capabilities with Amazon Kinesis Video Streams and Amazon Rekognition Video, Amazon has created a new set of services that can change how many organizations handle video moving forward.

#3 SageMaker

Amazon SageMaker is a fully managed end-to-end machine learning service that enables data scientists, developers, and machine learning experts to quickly build, train, and host machine learning models.

There are 3 components to SageMaker:

  • Creating Models: Using hosted Jupyter Notebooks, you can explore, clean and preprocess data.
  • Training Models: Train models with your own algorithms, or leverage the built-in algorithms or frameworks.
  • Hosting Models: A model hosting service with HTTPS endpoints to invoke your models in real-time.

One of the challenges we have seen in our work with machine learning in larger organizations is sharing models across all different groups that are interacting with it. SageMaker allows you to develop, document and share the models you create with other data scientists, and the authoring tool can also help you make the model understandable to non-experts.

The model hosting component facilitates accessibility to other applications. The SageMaker introduction article provides a pretty cool example of how this endpoint can now easily be invoked from Lambda and other channels, and showcases it with a twitter account that will guess to the location of an image.

#2 AWS Lambda Improvements and Cloud 9

A number of Lambda improvements were announced. At SapientRazorfish, we see big opportunities with AWS Lambda. This blog is managed through Lambda functions, and AWS Lambda is a Swiss Army knife in our DevOps practice. The major announcement around development was the collaborative development tool, Cloud 9.

Lambda development can still be fairly complicated. Especially in a team environment, ensuring all team members have the same configuration and test environment requires some work. With Cloud 9, all of this is now simplified, including an integrated debugger. Coding becomes the collaborative process it should be.

Beside Cloud 9, other announcements were also interesting. For starters, AWS Lambda has increased memory limits, and now provides the ability to manage concurrency, and supports execution throttling.

From a deployment perspective, Lambda now supports canary deployments to test a new version, shifting traffic based on pre-assigned weights to slowly move traffic to this new version as part of a broader rollout, and integration with CodeDeploy for full automation of this process.

The console has been improved and does a great job in vizualizing the configuration. The diagram below shows one of our DevOps utilities, a website performance monitor. The configuration visualizer shows that it is triggered by an event scheduler, and records the results on S3 and through CloudWatch.

The serverless application repository was also interesting. As mentioned, we leverage many smaller Lambda utilities in our DevOps practice. The creation of a serverless application repository can really help in sharing these with a broader audience. It will also be interesting to see what more complex serverless applications will be available through this mechanism.

We are very excited about all of these, it will simplify deployment and management of functions in AWS Lambda, and further increase adoption.

#1 Serverless Aurora

For me, the highlight of the conference was the announcement around serverless Aurora. With a serverless database, many applications can finally become fully serverless, with the associated benefits of straightforward scaling and true pay-as-you-go.

Although solutions like DynamoDB provided somewhat of an alternative, a serverless relational database opens up the opportunities for a far wider range of applications. With support for up to 64TB of storage, it can have broad implications. Within a cloud environment, database planning and sizing was still something that needed to be clearly planned upfront, and this has the potential to reduce that requirement significantly.

In a next post, I’ll talk about leveraging serverless Aurora.


With these announcements, Amazon is continuing to improve its platform, and it is exciting to see the opportunities it provides on changing our work.

written by: Martin Jacobs (VP, Technology)

Serverless Continuous Delivery for AEM on AWS

As mentioned in previous posts, I have been working on solutions that involve Adobe Experience Manager (AEM) and Amazon Web Services (AWS) to deliver digital experiences to consumers for the past few years. This was partly through custom implementations for our clients, and sometimes through Fluent, our turnkey digital marketing platform that combines AEM, AWS and Razorfish Managed Services and DevOps. I highlighted some of our ideas and best practices around managing a platform like that in a white paper created together with the AWS team.

One of the rules we apply in our work is striving for 100% automation in our deployment process. As I mention in the white paper, we leverage Jenkins on all our work to enable this. Combined with the fact that both AEM and AWS provide API access to various services and functionality, we have been very successful in automating this process.

That said, configuration of Jenkins and configuration of specific jobs is often a somewhat manual task. In addition, Jenkins (and the builds that are being executed) runs on its own EC2 instance(s), and therefore requires monitoring and incurring an ongoing cost. Especially in cases where deployments become less frequent, it would be ideal to utilize an on-demand & serverless deployment infrastructure.

AWS released CodePipeline earlier, and combined with the new AWS CodeBuild announcement, we can now look at replacing Jenkins by this set of AWS services, eliminating the need for a running AWS server that is a potential single point of failure. And with a unified architecture leveraging AWS tools, we have a solution that also integrates well with the overall AWS ecosystem, providing additional other benefits around artifact management and deployment.

I’ll describe how AWS Codepipeline can be used in the next section.


To describe the solution, a simple code pipeline is described, patterned after the WordPress example in the AWS documentation. It omits some critical additional steps, such a performance, security, regression, visual and functional tests, which I will touch upon in a subsequent post.

The we-retail sample application was used, a reference AEM application for a retail site developed by the Adobe team.

AWS CodePipeline Flow

As shown in the diagram, the CodePipeline connects to Github for the latest source code in the first step. It also retrieves configuration information and CloudFormation templates from AWS S3. If any updates to Github or to the S3 objects are made, CodePipeline will be triggered to rerun.

The next step is to compile the application. AWS CodeBuild is a perfect fit for this task, and a basic profile can be added to perform this compilation.

If compilation is successful, the next step is to create the staging infrastructure. We leverage our CloudFormation templates to instantiate this environment, and deploy the application into this. A manual approval step is then inserted to into the flow to ensure manual review and approval. Unfortunately, at this point it is not possible yet to include the CloudFormation output (the staging URL) into the approval message.

Once approval is gained, the staging infrastructure can be removed, and the application can be deployed to production. As production is an already existing environment, we will deploy the application using the standard API AEM provides for this. To do so, a Lambda job was created that mimics the set of curl commands that can be used to deploy a package and install it.


With AWS CodePipeline and CodeBuild, we can now create a fully automated serverless deployment platform.

Jenkins is still a very powerful solution. At this point, Jenkins also has a large number of plugins that help automate the various tasks. However, many of these can fairly easily be migrated to AWS CodePipeline. AWS CodePipeline already provides built in support for many different services, including BlazeMeter and HPE StormRunner. It also supports Jenkins integration itself, and for any other services, AWS Lambda can fill that gap. For example, in our original standard AEM on AWS setups, a large number of tasks (deployment, content transfer, etc) were performed by Jenkins invoking curl scripts, which can easily be implemented using AWS Lambda as shown in the example above.

Obviously, the example above is simplified. Additional steps for testing, content deployment or migration and code review will need to be incorporated. I will describe some of this in a next post. That said, creating this simple unified architecture is very powerful.

In addition, by leveraging the CodePipeline and CodeBuild architecture, the deployment process itself can be moved to a CloudFormation template, making it possible to instantiate a system with multiple environments with automated deployments with little more than a click of button, getting us closer to 100% automation.

written by: Martin Jacobs (GVP, Technology)

Play a YouTube Video When a Preview Image Is Clicked On Mobile


In this article, we will see how to play a YouTube video when a preview image is clicked or tapped, including on mobile. But before we get to that, let’s quickly recap the approaches that are often mentioned elsewhere, such as stack overflow, but do not work for mobile.

The most commonly cited method is to simply append ?autoplay=1 to the iframe src attribute. Another commonly proposed approach is to load the iframe api ( and call the method player.playVideo(). You may also have found comments, even marked as answers, concluding that it is impossible to achieve this user interface interaction on mobile browsers, such as mobile safari on iOS and chrome for android. Indeed, the aforementioned techniques will result in the YouTube player showing as stuck in a loading or buffering state on mobile browsers. There is a way to accomplish this interaction though–including on mobile–and it’s surprisingly simple.

Make the YouTube video transparent, and make it cover the preview image via z-index. When the user clicks or taps on the preview image, the user is actually clicking or tapping on the YouTube video, causing it to play. The final step is to listen for the player’s onStateChange event, and when the state changes to “playing”, hide or fade out your preview image and show or fade in the YouTube player.

See it in action:

written by: Dennis Hall (Presentation Layer Architect, Technology)

How Machine Learning Can Transform Content Management- Part II

In a previous post, I highlighted how machine learning can be applied to content management. In that article, I described how content analysis could identify social, emotional and language tone of an article. This informs a content author further on his writing style, and helps the author understand which style resonates most with his readers.

At the last AWS re:Invent conference, Amazon announced a new set of Artificial Intelligence Services, including natural language understanding (NLU), automatic speech recognition (ASR), visual search, image recognition and text-to-speech (TTS).

In previous posts, I discussed how the Amazon visual search and image recognition services could benefit digital asset management. This post will highlight the use of some of the other services in a content management scenario.

Text to Speech

From a content perspective, text to speech is a potential interesting addition to a content management system. One reason could be to assist visitors with vision problems. Screen readers can already perform many of these tasks, but leveraging the text to speech service provides publishers more control over the quality and tone of the speech.

Another use case is around story telling. One of the current trends is to create rich immersive stories, with extensive photography and visuals. This article from the New York Times describing the climb of El Capitan in Yosemite is a good example. With the new text to speech functionality, the article can easily be transformed into a self playing presentation with voice without any additional manual efforts, such as recruiting voice talent and the recording process.

Amazon Polly

Amazon Polly (the Amazon AI text to speech service) turns text into lifelike speech. It uses advanced deep learning technologies to synthesize speech that sounds like a human voice. Polly includes 47 lifelike voices that spread across 24 languages. Polly is service that can be used in real-time scenarios. It can used to retrieve a standard audio file (such as MP3) that can be stored and used at a later point. The lack of restrictions on storage and reuse of voice output makes it a great option for use in a content management system.

Polly and Adobe AEM Content Fragments

In our proof of concept, the Amazon AI text to speech service was applied to Content Fragments in Adobe AEM.

Content Fragment

As shown above, Content Fragments allow you to create channel-neutral content with (possibly channel-specific) variations. You can then use these fragments when creating pages. When creating a page, the content fragment can be broken up into separate paragraphs, and additional assets can be added at each paragraph break.

After creating a Content Fragment, the solution passes the content to Amazon Polly for retrieving the audio fragments with spoken content. It will generate a few files, one for the complete content fragment (main.mp3), and a set of files broken up by paragraph (main_1.mp3, main_2.mp3, main_3.mp3). The audio files are stored as associated content for the master Content Fragment.


When authoring a page that uses this Content Fragment, the audio fragments are visible in the sidebar, and can be added to the page if needed. With this capability in place, developing a custom story telling AEM component to support scenarios like the New York Times article becomes relatively simple.

If you want to explore the code, it is posted on Github as part of the repository.


This post highlighted how an old problem can now be addressed in a new way. The text to speech service will make creating more immersive experiences easier and more cost effective. Amazon Polly’s support for 24 languages opens up new possibilities as well. Besides the 2 examples mentioned in this post, it could also support scenarios such as interactive kiosks, museum tour guides and other IOT, multichannel or mobile experiences.

With these turnkey machine-learning services in place, creative and innovative thinking is critical to successfully solve challenges in new ways.

written by: Martin Jacobs (GVP, Technology)

How Machine Learning Can Transform Digital Asset Management - SmartCrop

In previous articles I discussed the opportunities for machine learning in digital asset management (DAM), and, as a proof of concept, integrated a DAM solution (Adobe AEM DAM with with various AI/ML solutions from Amazon, IBM, Google and Microsoft.

The primary use case for that proof of concept was around auto-tagging assets in a digital asset management solution. Better metadata makes it easier for authors, editors, and other users of the DAM to search for assets, and in some scenarios, the DAM can providing asset recommendations to content authors based on metadata. For example, it’s often important to have a diverse mix of people portrayed on your site. With gender, age, and other metadata attributes as part of the image, diversity can be enforced using asset recommendation or asset usage reports within a DAM or content management system.

Besides object recognition, the various vendors also provide API’s for facial analysis. Amazon AI for example provides a face analysis API, and this post will show how we can tackle a different use case with that service.


One common use case is the need for one image to be re-used at different sizes. A good example is the need for a small sized profile picture of the CEO for a company overview page as well as a larger version of that picture in the detailed bio page.

Challenge screenshot

Often, cropping at the center of an image often works fine, but it can also result in the wrong area being cropped. Resizing often distorts a picture, and ends up incorporating many irrelevant areas. A number of solutions are out there to deal with this problem, ranging from open source to proprietary vendors. All of them are leveraging different detection algorithms for identifying the area of interest in an image.


Leveraging the Amazon Rekognition Face Analysis API, we can now solve this problem in a very simple way. Using the API, a bounding box for the face can be retrieved, indicating the boundaries of a face in that picture. And with that bounding box, the right area for cropping can be identified. After cropping, any additional resizing can be done with the most relevant area of the image to ensure the image is at the requested size.

Solution screenshot

The result is shown in the image above. The image to the right is the result of leveraging the SmartCrop functionality based on the Face Analysis API. As you can see, it is a significant improvement over the other options. Improvements to this SmartCrop could be done by adding additional margin, or incorporating some of the additional elements retrieved by the Face Analysis API.

The code for this proof of concept is posted on the Razorfish Github account, as part of the repository. Obviously, in a real production scenario, additional optimizations should be performed to this proof of concept for overall performance reasons. The Amazon Rekognition API call only needs to take place once per image, and can potentially be done as part of the same auto-tagging workflow highlighted in previous posts, with the bounding box stored as an attribute with the image for later retrieval by the SmartCrop functionality. In addition, the output from the cropping can be cached at a CDN or webserver in front of Adobe AEM.


As this post highlights, old problems can be addressed now in new ways. In this case, it turns a task that often was performed manually in something that can be automated. The availability of many turnkey machine-learning services can provide a start to solve existing problems in a new and very simple manner. It will be interesting to see the developments in the coming year on this front.

written by: Martin Jacobs (GVP, Technology)

How Machine Learning Can Transform Digital Asset Management - Part III

In previous articles I discussed the opportunities for machine learning in digital asset management (DAM), and, as a proof of concept, integrated a DAM solution (Adobe AEM DAM with Google Cloud Vision. I followed up with a post on potential alternatives to Google’s Cloud Vision, including IBM Watson and Microsoft Cognitive Intelligence, and integrated those with Adobe DAM as well.

Of course, Amazon AWS couldn’t stay behind, and at the last AWS re:Invent conference, Amazon announced their set of Artificial Intelligence Services, including natural language understanding (NLU), automatic speech recognition (ASR), visual search and image recognition, text-to-speech (TTS). Obviously, it was now time to perform the integration with the AWS AI services in our proof of concept.

Amazon Rekognition

The first candidate for integration was Amazon Rekognition. Rekognition is a service that can detect objects, scenes, and faces in images and makes it easy to add image analysis to your applications. At this point, it offers 3 core services:

  • Object and scene detection - automatically labels objects, concepts and scenes
  • Facial analysis - analysis of facial attributes (e.g. emotion, gender, glasses, face bounding box)
  • Face comparison - compare faces to see how closely they match

Integration Approach

Google’s API was integrated using a SDK, the IBM and Microsoft API’s were integrated leveraging their standard REST interface. For the Amazon Rekognition integration, the SDK route was taken again, leveraging the AWS Java SDK. Once the SDK has been added to the project, the actual implementation becomes fairly straightforward.


From a digital asset management perspective, the previous posts focused on auto-tagging assets to support a content migration process or improve manual efforts performed by DAM users.

The object & scene detection for auto-tagging functioned well with Amazon Rekognition. However, the labels returned are generalized. For example, a picture of the Eiffel tower will be labeled “Tower” instead of recognizing the specific object.

The facial analysis API returns a broad set of attributes, including the location of facial landmarks such as mouth and nose. But it also includes attributes such as emotions and gender, which can be used as tags. These can then be beneficial in digital asset management scenarios such as search and targeting.

Many of the attributes and labels returned by the Rekognition API included a confidence score, indicating the confidence around a certain object detection.

Results screenshot

In the proof of concept, a 75% cut off was used. From the example above, you can see that Female, Smile and Happy have been detected as facial attributes with a higher than 75% confidence.


The source code and setup instructions for the integration with AWS, as well as Google, Microsoft, and IBM’s solutions can be found on Github in the Razorfish repository.

One thing all the different vendors have in common is that the services are very developer focused, and integrating these services with an application is very straightforward. This makes adoption easy. Hopefully, the objects being recognized will become more detailed and advanced over time, which will improve their applicability even more.

written by: Martin Jacobs (GVP, Technology)

Highlights from AWS re:Invent 2016

Another AWS re:Invent is behind us and it was packed with exciting announcements including the launch of new products, extension of existing services and much more. It was the biggest re:Invent ever with approximately a whopping 32,000 attendees and numerous exhibitors.

The conference was kicked off with a keynote from Andy Jassy, CEO of Amazon Web Services, who presented some impressive growth numbers and announced host of new updates to AWS portfolio of services. Biggest announcements were around new Artificial Intelligence (AI) services called Lex, Rekognition and Polly and data migrations appliances Snowmobile and Snowball Edge. He also launched Amazon Lightsail, which allows developers to setup a virtual private server (VPS) with just a few clicks.

The second keynote, presented by Amazon Web Services CTO Werner Vogels, was more focused on new development tools, Big Data, Security and Mobile services.

Here’s a rundown of the key announcements coming out of re:Invent this year.  

Amazon AI

One of the most significant announcement from Andy Jassy’s keynote was the launch of Amazon Lex, Amazon’s first AI service. Amazon Lex is a service for building conversational interfaces into any application using voice and text. It’s the technology that’s at the heart of the Amazon Alexa platform. This chat bot-friendly service is in preview.

Another AI service launched was Amazon Rekognition. Rekognition allows developers to add image analysis to applications. It can analyze and detect facial features and objects such as cars and furniture. Jassy also announced launch of Amazon Polly, which converts text into speech. Polly is a fully managed service and you can even cache responses making it cost efficient. It is available in 47 voices and 27 languages.  

Internet of Things (IoT)

AWS Greengrass is another interesting service launched at re:Invent. AWS Greengrass lets you run local compute, messaging & data caching for connected devices in a secure way. Greengrass seamlessly extends AWS to devices so they can act locally on the data they generate, while still using the cloud for management, analytics, and durable storage. It allows IoT devices to respond quickly to local events, operate with intermittent connections, and minimize the cost of transmitting IoT data to the cloud.

Data storage and services

Amazon Athena is an interactive query service that makes it easy to analyze data in Amazon’s Simple Storage Service (S3) using SQL. It is a great addition since it allows developers to use standard SQL syntax to query data that’s stored in S3 without setting up the infrastructure for it. This service works with CSV, JSON, log files, delimited files, and more.

Amazon Aurora, cloud-based relational database, now supports PostgreSQL. It’s already compatible with open source standards such as MySQL and MariaDB.


AWS Lambda, a serverless computing service, got a couple of updates as well. Amazon announced Lambda@Edge, the new Lambda-based processing model allows you to write code that runs within AWS edge locations. This lightweight request processing logic will handle requests and responses that flow through a CloudFront distribution. It is great for developers who need to automate simple tasks in their CDN deployment so that traffic does not have to be routed back to a server.

Lambda functions now includes support for the Microsoft’s C# programming language. It already supports Node.js, Python and Java. Amazon also unveiled AWS Step Functions as a way to create a visual state machine workflow out of your functions.  


As is tradition at re:Invent, Amazon announced a series of new core computing capabilities for its cloud. It launched F1 instances that support programmable hardware, R4 memory optimized instances, T2 burstable performance instances, compute-optimized C5 and I/O intensive I3 instances. Andy Jassy also announced Amazon EC2 Elastic GPUs, a way for people to attach GPU resources to EC2 instances. With Elastic GPUs for EC2 you can easily attach low-cost graphics acceleration to current generation EC2 instances.

Another important compute service launched is Amazon Lightsail. It allows developers to launch a virtual private server with just a few clicks. I think it is great addition to the portfolio as it allows small business owner and blogger to host their websites on AWS.  

Migration/ Data Transfer

Expanding on the scope of the Snowball which was launched last year, AWS added Snowball Edge and Snowmobile to the lineup. While Snowball provided 50TB of storage, each Snowball Edge appliance has 100TB of storage and offers more connectivity protocols than the previous version. Now you have also have Snowmobile to meet the needs of the customers with petabytes of data. Snowmobile is a 45-foot container that is delivered to customers on a trailer truck. This secure data truck stores up to 100 PB of data and can help companies move Exabyte of data to AWS in a matter of weeks instead of years. Snowmobile attaches to the clients network and appears as a local, NFS-mounted volume.  

Development tools

Amazon added AWS CodeBuild to the existing suite of developer tools like Code Commit, Code Deploy and Code Pipeline. AWS CodeBuild is a fully managed build service that compiles source code, runs tests, and produces software packages that are ready to deploy. CodeBuild can be a cost effective and scalable alternative to running a dedicated Jenkins instance.

AWS X-Ray helps developers analyze and debug production, distributed applications, such as those built using a micro services architecture. X-Ray provides an end-to-end view of requests as they travel through the application and helps developers identify and troubleshoot the root cause of performance issues and errors. AWS X-Ray is in preview.  

Monitoring Operations and Security

Similar to AWS Services Health Dashboard, AWS now provides a Personal Heath Dashboard. As the name indicates, this dashboard gives you a personalized view into the performance and availability of the AWS services that you are using, along with alerts that are automatically triggered by changes in the health of the services.

DDoS (Distributed Denial of Service) attacks are one very common trouble spot. Amazon new offering is AWS Shield, a DDoS protection service that safeguards web applications running on AWS. AWS Shield provides always-on detection and automatic inline mitigations that minimize application downtime and latency, so there is no need to engage AWS Support to benefit from DDoS protection. It provides DDoS protection at the DNS, CDN, and load balancer tiers and is available in free and premium flavors.  

Big Data and Compute

AWS Batch, a service for automating the deployment of batch processing jobs is released in preview. AWS Batch enables developers, administrators, and users to easily and efficiently run hundreds of thousands of batch computing jobs on AWS. With Batch users have access to the power of the cloud without having to provision, manage, monitor, or maintain clusters. No software to buy or install. AWS Glue a fully managed ETL service that makes it easy to move data between your data stores was also launched.

Mobile Services

Dr. Vogels also launched Amazon Pinpoint, a mobile analytics service. Amazon Pinpoint makes it easy to run targeted campaigns to drive user engagement in mobile apps through the use of targeted push notifications.

AWS refers to re:Invent as an educational event, and they were very successful in achieving this in 2016. You can find the recording of keynote and tech talks on YouTube.

written by: Praveen Modi (Sr Technical Architect)

AEM on AWS whitepaper

For a number of years now, I have been working on solutions that involve Adobe Experience Manager (AEM) and Amazon Web Services (AWS) to deliver digital experiences to consumers. Partly through custom implementations for our clients, and sometimes through Fluent, our turnkey digital marketing platform that combines AEM, AWS and Razorfish Managed Services and DevOps.

In the past year, I had to pleasure of working with AWS on creating a whitepaper that outlines some of our ideas and best practices around deploying on the AWS infrastructure.

The whitepaper delves into a few areas:

Why Use AEM on AWS?

Cloud-based solutions offer numerous advantages over on-premise infrastructure, particularly when it comes to flexibility.

For example, variations in traffic volume are a major challenge for content providers. After all, visitor levels can spike for special events such Black Friday Shopping, the Super Bowl, or other one-time occasions. AWS’s Cloud-based flexible capacity enables you to scale workloads up or down as needed.

Marketers and businesses utilize AEM as the foundation of their digital marketing platforms. Running it on AWS facilitates easy integration with third-party solutions, and make it a complete platform. Blogs, social media, and other auxiliary channels are simple to add and to integrate. In particular, using AEM in conjunction with AWS allows you to combine the API’s from each into powerful custom-configurations uniquely suited to your business needs. As a result, the tools for new features such as mobile delivery, analytics, and managing big data are at your fingertips.

A Look Under the Hood – Architecture, Features, and Services

In most cases, the AEM architecture involves three sets of services:

  1. Author – Used for the creation, management, and layout of the AEM experience.

  2. Publisher – Delivers the content experience to the intended audience. It includes the ability to personalize content and messaging to target audiences.

  3. Dispatcher – This is a caching and/or load-balancing tool that helps you realize a fast and performant experience for the end-user.

The whitepaper details some common configuration options for each service, and how to address that with the different storage options AEM provides. It also outlines some of the security considerations when deploying AEM in a certain configuration.

AWS Tools and Management

Not only does AWS allow you to build the right solution, it provides additional capabilities to make your environment more agile and secure.

Auditing and Security

The paper also highlights a couple of capabilities to make your platform more secure. For example, AWS has audit tools such as AWS Trusted Advisor. This tool automatically inspects your environment and makes recommendations that will help you cut costs, boost performance, improve, reliability, and enhance security. Other recommended tools include Amazon Inspector, which scans for vulnerabilities and deviations from best practices.

Automation through APIs

AWS provides API access to all its services and AEM does the same. This allows for a very clean organization of the integration and deployment process. Used in conjunction with an automated open-source server like Jenkins you can initiate manual, scheduled, or triggered deployments.

You can fully automate the deployment process. However, depending on which data storage options are used, separate arrangements may need to be made for backing up data. Also, policies and procedures for dealing with data loss and recovery need to be considered too.

Additional AWS Services

There are numerous other services and capabilities you can leverage when you use AEM in conjunction with AEM.

One great service is Amazon CloudWatch, which allows you to monitor a variety of performance metrics from a single place.

In addition, the constant stream of new AWS offerings, such as, Amazon’s Elastic File System (which allows you to configure file storage for your servers), provide new options for your platform.


Using Adobe’s Experience Manager in tandem with AWS provides a powerful platform for easily delivering highly immersive and engaging digital experiences. It is a solution that answers the dilemma that many marketers and content providers face – how to deliver an immersive, relevant, and seamless experience for end users from a unified platform, and the whitepaper aims to provide more insight into how to achieve this.

To learn more about running AEM on AWS, you can download and read the full whitepaper here.

written by: Martin Jacobs (GVP, Technology)