Architecture Archive

AEM on AWS whitepaper

For a number of years now, I have been working on solutions that involve Adobe Experience Manager (AEM) and Amazon Web Services (AWS) to deliver digital experiences to consumers. Partly through custom implementations for our clients, and sometimes through Fluent, our turnkey digital marketing platform that combines AEM, AWS and Razorfish Managed Services and DevOps.

In the past year, I had to pleasure of working with AWS on creating a whitepaper that outlines some of our ideas and best practices around deploying on the AWS infrastructure.

The whitepaper delves into a few areas:

Why Use AEM on AWS?

Cloud-based solutions offer numerous advantages over on-premise infrastructure, particularly when it comes to flexibility.

For example, variations in traffic volume are a major challenge for content providers. After all, visitor levels can spike for special events such Black Friday Shopping, the Super Bowl, or other one-time occasions. AWS’s Cloud-based flexible capacity enables you to scale workloads up or down as needed.

Marketers and businesses utilize AEM as the foundation of their digital marketing platforms. Running it on AWS facilitates easy integration with third-party solutions, and make it a complete platform. Blogs, social media, and other auxiliary channels are simple to add and to integrate. In particular, using AEM in conjunction with AWS allows you to combine the API’s from each into powerful custom-configurations uniquely suited to your business needs. As a result, the tools for new features such as mobile delivery, analytics, and managing big data are at your fingertips.

A Look Under the Hood – Architecture, Features, and Services

In most cases, the AEM architecture involves three sets of services:

  1. Author – Used for the creation, management, and layout of the AEM experience.

  2. Publisher – Delivers the content experience to the intended audience. It includes the ability to personalize content and messaging to target audiences.

  3. Dispatcher – This is a caching and/or load-balancing tool that helps you realize a fast and performant experience for the end-user.

The whitepaper details some common configuration options for each service, and how to address that with the different storage options AEM provides. It also outlines some of the security considerations when deploying AEM in a certain configuration.

AWS Tools and Management

Not only does AWS allow you to build the right solution, it provides additional capabilities to make your environment more agile and secure.

Auditing and Security

The paper also highlights a couple of capabilities to make your platform more secure. For example, AWS has audit tools such as AWS Trusted Advisor. This tool automatically inspects your environment and makes recommendations that will help you cut costs, boost performance, improve, reliability, and enhance security. Other recommended tools include Amazon Inspector, which scans for vulnerabilities and deviations from best practices.

Automation through APIs

AWS provides API access to all its services and AEM does the same. This allows for a very clean organization of the integration and deployment process. Used in conjunction with an automated open-source server like Jenkins you can initiate manual, scheduled, or triggered deployments.

You can fully automate the deployment process. However, depending on which data storage options are used, separate arrangements may need to be made for backing up data. Also, policies and procedures for dealing with data loss and recovery need to be considered too.

Additional AWS Services

There are numerous other services and capabilities you can leverage when you use AEM in conjunction with AEM.

One great service is Amazon CloudWatch, which allows you to monitor a variety of performance metrics from a single place.

In addition, the constant stream of new AWS offerings, such as, Amazon’s Elastic File System (which allows you to configure file storage for your servers), provide new options for your platform.

Takeaway

Using Adobe’s Experience Manager in tandem with AWS provides a powerful platform for easily delivering highly immersive and engaging digital experiences. It is a solution that answers the dilemma that many marketers and content providers face – how to deliver an immersive, relevant, and seamless experience for end users from a unified platform, and the whitepaper aims to provide more insight into how to achieve this.

To learn more about running AEM on AWS, you can download and read the full whitepaper here.

written by: Martin Jacobs (GVP, Technology)

Lessons Learned from a Reactive Serverless CMS

Background

As mentioned in previous posts, we are big proponents of reactive architectures at Razorfish.

We also believe architectures using cloud functions — such as AWS Lambda — are part of the future of application development. In this post, we will call them “serverless” architectures because although there are obviously still servers involved, we’re not responsible for managing them anymore.

The relaunch of our technology blog provided the perfect opportunity to test this new architecture. In the paragraphs that follow, I’ll briefly discuss the architecture, followed by a summary of the lessons we learned.

Solution Summary

We architected the solution using Amazon AWS S3, Lambda, Cloudfront, Hugo, and Github. It incorporates an authoring UI, as well as a mechanism to do publishing. The diagram below shows some of the integration mechanisms. For the full technical details of the implementation, visit the earlier post on the Razorfish technology blog.

Learning — Serverless: Development Model

Obviously, development using AWS Lambda is quite different than your standard processes. But there’s good news: A large number of new tools are trying to address this, and we explored a few of them. Some of the most interesting include:

  • Lambda-local. This is a basic command line tool you can use to run the Amazon Lambda function on local machines.
  • Node-Lambda. Similar to Lambda, this tool provides support for deploying a function to AWS.
  • Apex. This large framework can be used to deploy lambda functions, potentially written in additional languages such as Go — which Apex itself is written in. The tool provides support for Terraform to manage AWS resources.
  • Kappa — Another tool for deployment of Lambda functions, using the AWS API for creation of resources.
  • Serverless. An application framework for building applications using AWS Lambda and API Gateway. It tries to streamline the development of a microservices-based application. It creates AWS resources using CloudFormation templates, giving you the benefits of tracking and managing resource creation. It also supports different types of plugins, allowing you to quickly add additional capabilities to an application (e.g., logging). One of the objectives of the tool is to support multiple cloud providers, including Google and Azure Cloud Functions.
  • λ Gordon — Similar to Apex, a solution to create and deploy lambda functions, using CloudFormation to manage these resources, with a solid set of example functions.
  • Zappa. Zappa allows you to deploy Python WSGI applications on AWS Lambda + API Gateway. Django and Flask are examples of WSGI applications that can now be deployed on AWS Lambda using Flask-Zappa or Django-Zappa. In addition to these tools, IDE’s have developed ways to make it easier to create and deploy lambda functions. For example, Visual Studio and Eclipse have tools to make it easier to create, test, and deploy functions.

Lambda-local was the tool of choice for the serverless CMS application created for our blog. Its simplicity is helpful, and one of the unique challenges we faced was the support needed for binaries like Hugo and Libgit2, which required development both on the local machines and on an Amazon EC2 Linux instance.

Learning — Serverless: Execution Model

Although the initial use cases for AWS Lambda and other similar solutions have been styled around executing backend tasks like image resizing, interactive web applications can become an option as well.

For a start, many solutions don’t necessarily need to be a server side web application, and can often be architected as a static using client-side JavaScript for dynamic functionality. So in the AWS scenario, this means a site hosted on S3 or Cloudfront and then integrate with AWS Lambda using the JavaScript SDK or the API gateway — similar to how this was done for the Razorfish blog.

But in case the dynamic element is more complex, there is a great potential for full-featured frameworks like Zappa that allow you to develop interactive web applications that can run on AWS Lambda using common frameworks such as Django and Flask. In my opinion, this is also where AWS can get significant competition from Azure Functions, as Microsoft has an opportunity to create very powerful tools with their Visual Studio solution.

Overall, AWS Lambda is a great fit for many types of applications. The tool significantly simplifies the management of applications; there’s limited need to perform ongoing server monitoring and management that is required with AWS EC2 or AWS Elastic Beanstalk.

On top of that, Lambda is incredibly affordable. As an example, if you required 128MB of memory for your function, executed it 30 million times in one month for 200ms each time, your monthly bill would be $11.63 — which is cheaper than running most EC2 instances.

The Razorfish technology blog architecture is network intensive. It retrieves and uploads content from S3 or Github. With AWS Lambda, you choose the amount of memory you want to allocate to your functions and AWS Lambda allocates proportional CPU power, network bandwidth, and disk I/O. So in this case, an increase in memory was needed to ensure enough bandwidth for the Lambda functions to execute in time.

Learning — Reactive flows

The second goal of the creation of our blog was to apply a reactive architecture. A refresher: Reactive programming is a programming style oriented around data flows and the propagation of change. Its primary style is asynchronous message passing between components of the solution, which ensures loose coupling.

In the blog scenario, this was primarily achieved by using S3 events, Github hooks, and SNS message passing. Some examples:

  • When one Lambda function was finished, an SNS message was published for another Lambda function to execute.
  • Client-side content updates are posted to S3, and the S3 event generated triggered Lambda functions.
  • A Github update posts to SNS, and the SNS triggers a Lambda function.

Overall, this allowed for a very simple architecture. It also makes it very straightforward to test and validate parts of the solution in isolation.

One of the key challenges, however, is rooted in the fact that there are potential scenarios where it becomes difficult to keep track of all different events and resulting messages generated. This can potentially result in loops or cascading results.

The Developer’s Takeaway

Overall, I believe the architectural style of reactive and serverless has a lot of promise — and may even be transformational with respect to developing applications in the future. The benefits are tremendous, and will allow us to really take cloud architectures to the next level. For this reason alone, developers should consider how this approach could be incorporated into every new project.

written by: Martin Jacobs (GVP, Technology)

Sitecore with React and SEO

On a recent Sitecore build-out, the architecture included, among other things, a significant amount of functionality that was provided by a client-side accessed API.  We chose React to connect to and provide the UI for that API.  Since we were already using React for some things, we chose to standardize on React and use it for all of the UI.

The prevailing approach to be found around the web was to write the entire page as a single React component.  The rare article or guide that spoke of using smaller standalone components all suggested supplying the components with their data in the form of JSON.

Let’s take a look at the example of a simple, standalone component on React’s homepage:

On the last line (line 7) above, we see that the property value, subject="World", is being supplied to the component. But let’s assume that "World" is text that is managed by the CMS.  How would we get that text from the CMS to React? Popular thinking suggests outputting all the data you would need as JSON–and passed directly to React.render, inside of a script tag, much as you see on line 22 below.

The problem is, that approach doesn’t provide very good SEO value.

So how can we provide the data to our components in an SEO-friendly way? Interestingly, use one of the technologies introduced to assist search engines to improve the semantic value of the data they crawl–schema.org.

Schema.org is a collaborative, community activity with a mission to create, maintain, and promote schemas for structured data on the Internet, on web pages, in email messages, and beyond.

Schema.org vocabulary can be used with many different encodings, including RDFa, Microdata and JSON-LD. These vocabularies cover entities, relationships between entities and actions, and can easily be extended through a well-documented extension model.

Schema.org markup includes the concepts of itemscope, and itemprop.  We adapted this approach to provide a markup structure and attributes that would readily convert to the JSON-full-of-component-props that we need to render React components.

A simplified version of the final result:

As you can see, data-react="%ReactComponentName%" attribute-value pair identifies a markup structure as a React component.  Inside this structure, data-prop="%propertyName%" attribute-value signifies that the contents of a given html tag represent the value of that property. For instance, data-react="HelloMessage" identifies a markup structure as representing the place in the page where a HelloMessage component should render. And the markup structure also contains data-prop’s to provide the props data for the component. The first HelloMessage has data-prop="greeting" with text contents of "Hello". This is converted to { greeting: "Hello" } before being passed in when rendering the component.

Consider the following markup:

The above markup gets converted to the JavaScript object below:

As an added convenience, the React components are not rendered into some other DOM Node, as in the previous examples. Instead, quite naturally, they are rendered in-place right where they are defined in the markup, using the props defined inside the same markup.

And there we have it, Sitecore with React and SEO.

written by: Dennis Hall (Presentation Layer Architect, Technology)

A reactive serverless cms for the technology blog

Background

At Razorfish, we are big proponents of reactive architectures. Additionally, we believe architectures using cloud functions such as AWS Lambda are part of the future of application development. Our relaunch of the blog was a good occasion to test this out.

Historically, the blog had been hosted on WordPress. Although WordPress is a good solution, we had run into some performance issues. Although there are many good ways to address performance challenges in WordPress, it was a good time to explore a new architecture for the blog, as we weren’t utilizing any WordPress specific features.

We had used static site generators for a while for other initiatives, and looked at these types of solutions to create the new site. We wanted to avoid any running servers, either locally or in the cloud.

Our technology stack ended up as follows:

  • Github – Contains two repositories, a content repository with Hugo based themes, layout and content, and a code repository with all the cms code.

  • AWS Lambda

  • Hugo – Site Generator written in the Go Programming Language

  • AWS S3 – Source and generated sites are stored on S3

  • AWS CloudFront – CDN for delivery of site.

Why Hugo?

There are a large number of site generators available, ranging from Jekyll to Middleman. We explored many of them, and decided on Hugo for a couple of reasons:

  • Speed – Hugo generation happens in seconds

  • Simplicity - Hugo is very easy to install and run. It is a single executable, without any dependencies

  • Structure - Hugo has a flexible structure, allowing you to go beyond blogs.

Architecture

The architecture is outlined below. A number of Lambda functions are responsible for executing the different functions of the CMS. Some of the use of Hugo was derived from http://bezdelev.com/post/hugo-aws-lambda-static-website/. The authentication function was loosely derived from https://github.com/danilop/LambdAuth.

The solution uses AWS lambda capabilities to run executables. This is used for invoking Hugo, but also for incorporating libgit2, which allows us to execute git commands and integrate with Github.

CMS

As part of the solution, a CMS UI was developed to manage content. It allows the author to create new content, upload new assets, and make other changes.

Content is expected to be in Markdown format, but this is simplified for authors with the help of the hallojs editor.

Preview is supported with different breakpoints, including a mobile view.

As it was developed as a reactive architecture, other ways to update content are available:

  • Through a commit on github, potentially using github’s markdown editor.

  • Upload or edit markdown files directly on S3

Challenges

As the solution was architected, a few interesting challenges had to be addressed.

  1. At development, only Node 0.14 was supported on AWS. To utilize solutions like libgit2, a more recent version of Node was needed. To do so, a Node executable was packaged as part of the deploy, and Node 0.14 spawned the more recent Node version.

  2. Only the actual site should be accessible. To prevent preview and other environments from being accessible, CloudFront signed cookies provided a mechanism to prevent the other environments from being directly accessible.

  3. Hugo and libgit are libraries that need to be compiled for the AWS Lambda linux environment, which can be a challenge with all other development occurring on Windows or Macs.

Architecture benefits

The reactive architecture approach makes it really easy to enhance and extend the solution with other options of integrating content or experience features.

For example, as an alternative to the described content editing solutions above, other options can be identified:

  • A headless CMS like Contentful could be added for a richer authoring UI experience.

  • By using SES and Lambda to receive and process the email, an email content creation flow could be setup.

  • A convertor like pandoc on AWS Lambda can be incorporated into the authoring flow, for example for converting source documents to the target markdown format. It possibly can be invoked from the CMS UI, or from the email processor.

From an end-user experience perspective, Disqus or other 3rd party providers are obvious examples to incorporate comments. However, the lambda architecture can also be an option to easily add commenting functionality.

Conclusion

Although more and more tools are coming available, AWS Lambda development and code management can still be a challenge, especially in our scenario with OS specific executables. However, from an architecture perspective, the solution is working very well. It has become very predictive and stable, and allows for a fully hands-off approach on management.

written by: Martin Jacobs (GVP, Technology)