Sitecore with React and SEO

On a recent Sitecore build-out, the architecture included, among other things, a significant amount of functionality that was provided by a client-side accessed API.  We chose React to connect to and provide the UI for that API.  Since we were already using React for some things, we chose to standardize on React and use it for all of the UI.

The prevailing approach to be found around the web was to write the entire page as a single React component.  The rare article or guide that spoke of using smaller standalone components all suggested supplying the components with their data in the form of JSON.

Let’s take a look at the example of a simple, standalone component on React’s homepage:

On the last line (line 7) above, we see that the property value, subject="World", is being supplied to the component. But let’s assume that "World" is text that is managed by the CMS.  How would we get that text from the CMS to React? Popular thinking suggests outputting all the data you would need as JSON–and passed directly to React.render, inside of a script tag, much as you see on line 22 below.

The problem is, that approach doesn’t provide very good SEO value.

So how can we provide the data to our components in an SEO-friendly way? Interestingly, use one of the technologies introduced to assist search engines to improve the semantic value of the data they crawl–schema.org.

Schema.org is a collaborative, community activity with a mission to create, maintain, and promote schemas for structured data on the Internet, on web pages, in email messages, and beyond.

Schema.org vocabulary can be used with many different encodings, including RDFa, Microdata and JSON-LD. These vocabularies cover entities, relationships between entities and actions, and can easily be extended through a well-documented extension model.

Schema.org markup includes the concepts of itemscope, and itemprop.  We adapted this approach to provide a markup structure and attributes that would readily convert to the JSON-full-of-component-props that we need to render React components.

A simplified version of the final result:

As you can see, data-react="%ReactComponentName%" attribute-value pair identifies a markup structure as a React component.  Inside this structure, data-prop="%propertyName%" attribute-value signifies that the contents of a given html tag represent the value of that property. For instance, data-react="HelloMessage" identifies a markup structure as representing the place in the page where a HelloMessage component should render. And the markup structure also contains data-prop’s to provide the props data for the component. The first HelloMessage has data-prop="greeting" with text contents of "Hello". This is converted to { greeting: "Hello" } before being passed in when rendering the component.

Consider the following markup:

The above markup gets converted to the JavaScript object below:

As an added convenience, the React components are not rendered into some other DOM Node, as in the previous examples. Instead, quite naturally, they are rendered in-place right where they are defined in the markup, using the props defined inside the same markup.

And there we have it, Sitecore with React and SEO.

written by: Dennis Hall (Presentation Layer Architect, Technology)

What the Rise of Cloud Computing Means for Infrastructure

Infrastructure setup and application programming are merging into simultaneous processes. With this critical change, we need to take a fresh look at how we design solutions. If we don’t, our projects risk failure.

Building and installing system infrastructure (think servers and networks) was once an arduous process. Everything had to be planned out and procured, often at high costs and with a long lead time. Often times, server specifications were created before the actual application (and the technologies involved) that would need to run on it had been fully flushed out. The actual application programming task was a whole separate step with little overlap.

That’s no longer the case due the rise of Cloud computing. Infrastructure is now software, and the convenience of that leads to new challenges.

Merging Designs

With Cloud computing, Infrastructure is way more fluid thanks to all the programmable elements. As a result, upfront planning isn’t as important, as cost and especially timelines are not a constraint anymore. Compute, storage and network capacity is immediately accessible and can be changed dynamically to suit any need.

With these changes, the days of separate tracks for application and infrastructure development are over. The once separate design processes for each of them need to merge as well. This is largely driven by 3 factors:

  1. Historically, the separation of application and infrastructure development didn’t work, but it was accepted as a given.
  2. Cloud architectures take on a bigger role than traditional infrastructure
  3. New Architectures create new demands

The Historical Challenge

Performance, availability and scalability have been a challenge forever. Before cloud architectures became standard, vendors have been trying to address these requirements with complex caching architectures, and similar mechanisms. The reality is that none of the products really delivered on this premise out of the box. Obviously, one core challenges was that companies were trying to deliver dynamic experiences on a fixed infrastructure.

But even within that fixed infrastructure, any deployment required exhaustive performance tuning cycles and vendor support, trying to overcome the issue of infrastructure independently designed from the application, with only moderate success.

The Changing Infrastructure Role

Cloud architectures also start to play a bigger role in the overall systems stack. Let’s look at a hypothetical basic Java application with an API build on Amazon Web Services, the most popular cloud computing service, to see what the merger of system infrastructure and application programming looks like.

The application can be developed like any other Java application, but once it comes to how security is addressed, what is specified where?

On the application side, there could be some internal security mechanisms that define what access to services is available. Internal application roles can determine what access to different data elements the service request has. From an infrastructure perspective, Amazon Web Services can also provide security measures (access to ports, another layer of permissions, etc.) that affect how the application API can be accessed by clients. In addition, Amazon’s AWS policies can define which request arrives at the application, or which data elements are available once a request is being serviced.

As this example shows, the application and infrastructure views need to be merged in order to fully understand the security mechanisms available. Just focusing on one side or the other paints an unclear picture.

New Architectures

A number of new architectures have been created now that infrastructure is programmable. Reactive architectures and code executors like Google Cloud Functions and AWS Lambda are examples of these serverless computing services. Once we start using fully dynamic infrastructures for auto-scaling and micro services, the need for in integrated view of both the application and systems becomes even more important.

Finding New Solutions

Handling infrastructure and application development in an integrated manner is difficult.

One of the challenge is that the design tools to visualize this are lacking. Tools like Cloudcraft help in this regard but a fully integrated view is lacking, especially if you start using new architectures like AWS Lambda. Ideally, there’d be a way to visually layer the different perspectives of an architecture in a way that resembles a Photoshop image. Easily looking at an architecture from the perspective of security, services, data flows, and so on would be incredibly useful.

From a process perspective, infrastructure and application have to be handled with the same processes. This includes code management, defect tracking and deployment. This of course has implications on the skills and technology needed to successfully complete a project, and not all organizations are ready for this yet.

Conclusion

These days, infrastructure and application are intertwined, and an application solution that doesn’t address the infrastructure element is incomplete. Focusing on one without the other cannot address the critical requirements around security, performance, scalability, availability and others. It is important to invest in the tools, processes and people to deliver on this.

written by: Martin Jacobs (GVP, Technology)

A reactive serverless cms for the technology blog

Background

At Razorfish, we are big proponents of reactive architectures. Additionally, we believe architectures using cloud functions such as AWS Lambda are part of the future of application development. Our relaunch of the blog was a good occasion to test this out.

Historically, the blog had been hosted on WordPress. Although WordPress is a good solution, we had run into some performance issues. Although there are many good ways to address performance challenges in WordPress, it was a good time to explore a new architecture for the blog, as we weren’t utilizing any WordPress specific features.

We had used static site generators for a while for other initiatives, and looked at these types of solutions to create the new site. We wanted to avoid any running servers, either locally or in the cloud.

Our technology stack ended up as follows:

  • Github – Contains two repositories, a content repository with Hugo based themes, layout and content, and a code repository with all the cms code.

  • AWS Lambda

  • Hugo – Site Generator written in the Go Programming Language

  • AWS S3 – Source and generated sites are stored on S3

  • AWS CloudFront – CDN for delivery of site.

Why Hugo?

There are a large number of site generators available, ranging from Jekyll to Middleman. We explored many of them, and decided on Hugo for a couple of reasons:

  • Speed – Hugo generation happens in seconds

  • Simplicity - Hugo is very easy to install and run. It is a single executable, without any dependencies

  • Structure - Hugo has a flexible structure, allowing you to go beyond blogs.

Architecture

The architecture is outlined below. A number of Lambda functions are responsible for executing the different functions of the CMS. Some of the use of Hugo was derived from http://bezdelev.com/post/hugo-aws-lambda-static-website/. The authentication function was loosely derived from https://github.com/danilop/LambdAuth.

The solution uses AWS lambda capabilities to run executables. This is used for invoking Hugo, but also for incorporating libgit2, which allows us to execute git commands and integrate with Github.

CMS

As part of the solution, a CMS UI was developed to manage content. It allows the author to create new content, upload new assets, and make other changes.

Content is expected to be in Markdown format, but this is simplified for authors with the help of the hallojs editor.

Preview is supported with different breakpoints, including a mobile view.

As it was developed as a reactive architecture, other ways to update content are available:

  • Through a commit on github, potentially using github’s markdown editor.

  • Upload or edit markdown files directly on S3

Challenges

As the solution was architected, a few interesting challenges had to be addressed.

  1. At development, only Node 0.14 was supported on AWS. To utilize solutions like libgit2, a more recent version of Node was needed. To do so, a Node executable was packaged as part of the deploy, and Node 0.14 spawned the more recent Node version.

  2. Only the actual site should be accessible. To prevent preview and other environments from being accessible, CloudFront signed cookies provided a mechanism to prevent the other environments from being directly accessible.

  3. Hugo and libgit are libraries that need to be compiled for the AWS Lambda linux environment, which can be a challenge with all other development occurring on Windows or Macs.

Architecture benefits

The reactive architecture approach makes it really easy to enhance and extend the solution with other options of integrating content or experience features.

For example, as an alternative to the described content editing solutions above, other options can be identified:

  • A headless CMS like Contentful could be added for a richer authoring UI experience.

  • By using SES and Lambda to receive and process the email, an email content creation flow could be setup.

  • A convertor like pandoc on AWS Lambda can be incorporated into the authoring flow, for example for converting source documents to the target markdown format. It possibly can be invoked from the CMS UI, or from the email processor.

From an end-user experience perspective, Disqus or other 3rd party providers are obvious examples to incorporate comments. However, the lambda architecture can also be an option to easily add commenting functionality.

Conclusion

Although more and more tools are coming available, AWS Lambda development and code management can still be a challenge, especially in our scenario with OS specific executables. However, from an architecture perspective, the solution is working very well. It has become very predictive and stable, and allows for a fully hands-off approach on management.

written by: Martin Jacobs (GVP, Technology)

How Machine Learning Can Transform Content Management

In previous posts, I explored the opportunities for machine learning in digital asset management, and, as a proof-of-concept, integrated a DAM solution (Adobe AEM DAM) with a set of machine learning APIs.

But the scope of machine learning extends much further. Machine learning can also have a profoundly positive impact on content management.

In this context, machine learning is usually associated with content delivery. The technology can be used to deliver personalized or targeted content to website visitors and other content consumers. Although this is important, I believe there is another opportunity that stems from incorporating machine learning into the content creation process.

Questions Machine Learning Can Answer

During the content creation process, content can be analyzed by machine learning algorithms to help address some key questions:

  • How does content come across to readers? Which tones resonate the most? What writer is successful with which tone? Tone analysis can help answer that.
  • What topics are covered most frequently? How much duplication exists? Text clustering can help you analyze your overall content repository.
  • What is the article about? Summarization can extract relevant points and topics from a piece of text, potentially helping you create headlines.
  • What are the key topics covered in this article? You can use automatic topic extraction and text classification to create metadata for the article to make it more linkable and findable (both within the content management tool internally and via search engines externally).

Now that we know how machine learning can transform the content creation process, let’s take a look at a specific example of the technology in action.

An Implementation with AEM 6.2 Content Fragments and IBM Watson

Adobe just released a new version of Experience Manager, which I discussed in a previous post. One of the more important features of AEM 6.2 is the concept of “Content Fragments.”

In the past, content was often tied to pages. But Content Fragments allow you to create channel-neutral content with (possibly channel-specific) variations. You can then use these fragments when creating pages.

Content Fragments are treated as assets, which makes them great candidates for applying analysis and classification. Using machine learning, we’re able to analyze the tone of each piece of content. Tones can then be associated with specific pieces of content.

In the implementation, we used IBM Bluemix APIs to perform tone analysis. The Tone Analyzer computes emotional tone (joy, fear, sadness, disgust or anger), social tone (openness, conscientiousness, extraversion, agreeableness or emotional range) and language tone (analytical, confident or tentative).

The Tone Analyzer also provides insight on how the content is coming across to readers. For each sub-tone, it provides a score between 0 and 1. In our implementation, we associated a sub-tone with the metadata only if the score was 0.75 or higher.

The Results of Our Implementation

If you want to take a look, you’ll find the source code and setup instructions for the integration of Content Fragments with IBM Watson Bluemix over on GitHub.

We ran our implementation against the text of Steve Jobs’ 2005 Stanford commencement speech, and the results are shown below. For every sub-tone with a score of 0.75 or higher, a metadata tag was added.

Results

The Takeaway from Our Implementation

Machine learning provides a lot of opportunities during the content creation process. But it also requires authoring and editing tools that seamlessly integrate with these capabilities to really drive adoption and make the use of those machine-learning insights a common practice.

Personally, I can’t wait to analyze my own posts from the last few months using this technology. That analysis, combined with LinkedIn analytics, will enable me to explore how I can improve my own writing and make it more effective.

Isn’t that what every content creator wants?

written by: Martin Jacobs (GVP, Technology)

How Machine Learning Can Transform Digital Asset Management - Part II

A few weeks ago, I discussed the opportunities for machine learning in digital asset management (DAM), and, as a proof of concept, integrated a DAM solution (Adobe AEM DAM) with Google Cloud Vision, a newly released set of APIs for image recognition and classification.

Now, let’s explore some alternatives to Cloud Vision.

IBM Watson

To follow up, we integrated IBM’s offering. As part of BlueMix, IBM actually has two sets of APIs: the AlchemyAPI (acquired in March 2015) and the Visual Recognition API. The ability to train your own custom classifier in the Visual Recognition API is the key difference between the two.

There are a number of APIs within the AlchemyAPI, including a Face Detection/Recognition API and an Image Tagging API. The Face API includes celebrity detection and disambiguation of a particular celebrity (e.g., which Jason Alexander?).

Result sets can provide an age range for the identified people in the image. In a DAM scenario, getting a range instead of a number can be particularly helpful. The ability to create your own customer classifier could be very valuable with respect to creating accurate results for your specific domain. For example, you could create a trained model for your products, and organize your brand assets automatically against this. It would enable to further analyze usage and impact of these assets across new and different dimensions.

Leveraging the APIs was fairly straightforward. Similar to Google, adding the API to an application is simple; just build upon the sample API provided by IBM. It’s worth noting, however, that IBM’s API has a 1MB image size restrictions, somewhat lower than Google and Microsoft’s 4MB limit.

Microsoft Cognitive Services

Microsoft is interesting, especially considering they won the most recent ImageNet Large Scale Visual Recognition Challenge. As part of its Cognitive Services offering, Microsoft released a set of applicable Vision APIs (though they’re still in preview mode). For our purposes, the most relevant APIs are:

  • Computer Vision: This API incorporates an ability to analyze images and derive the appropriate tags with their confidence score. It can detect adult and racy content, and similar to Google’s Cloud Vision API, it has an Optical Character Recognition (OCR) capability that reads text in images. Besides tags, the API can provide English language descriptions of an image — written in complete sentences. It also supports the concepts of models. The first model is celebrity recognition, although we couldn’t get that one to work for straightforward celebrities like Barack Obama and Lionel Messi (it also doesn’t seem to work on the landing page).
  • Emotion: This API uses a facial expression in an image as an input. It returns the confidence level across a set of emotions for each face in relevant images.
  • Face: This API is particularly interesting, as it allows you to perform face recognition within a self-defined group. In a DAM scenario, this could be very relevant. For example, when all product images are shot with a small set of models, it can easily and more accurately classify each image with respect to various models. If an organization has contracts with a small set of celebrities for advertising prints, classification becomes that much more accurate.

The Microsoft APIs are dependent on each other in certain scenarios. For example, the Emotion API leverages the Face API to first identify faces within an image. Similarly, the Computer Vision API and Face API both identify gender and other attributes of people within an image.

Although Microsoft didn’t provide a sample Java API, the REST API is easy to incorporate. The source code and setup instructions for the integration with Google, Microsoft, and IBM’s solution can be found on Github.

Adobe Smart Tags

At the recent Adobe Summit conference, Adobe also announced the use of machine intelligence for smart tagging of assets as a beta capability of their new AEM 6.2 release. According to Adobe, it can automatically tag images with keywords based on:

  • Photo type (macro, portrait, etc.)
  • Popular activities (running, skiing, hiking, etc.)
  • Certain emotions (smiling, crying, etc.)
  • Popular objects (cars, roads, people, etc.)
  • Animals (dogs, cats, bears, etc.)
  • Popular locations (New York City, Paris, San Francisco, etc.)
  • Primary colors (red, blue, green)

There are even more categories for automatic classification, too.

Automatic Tagging Use Cases

In the previous post, I highlighted a couple of key use cases for tagging using machine intelligence in DAM. In particular, I highlighted how tagging can support the content migration process or improve manual efforts performed by DAM users.

Better metadata makes it easier for authors, editors, and other users of the DAM to find content during the creation process. It can also help in providing asset recommendations to content authors. For example, it’s often important to have a diverse mix of people portrayed on your site. With gender, age, and other metadata attributes as part of the image, diversity can be enforced using asset recommendation or asset usage reports within a DAM or content management system.

What’s more, this metadata can also help improve targeting and effectiveness of the actual end-user experience by:

  1. Allowing the image to be selected as targeted content
  2. Using the metadata in an image to ensure relevant ads, content, and assets are presented in context within an asset
  3. Informing site analytics by incorporating image metadata in click tracking and other measurement tools

In addition to these use cases, new scenarios are being created. Microsoft automatically generates captions your photos. Facebook is using machine intelligence to automatically assign alt text to photos uploaded to Facebook, and, in doing so, improve overall accessibility for Facebook users. Obviously, this type of functionality also will also enable Facebook and Microsoft to provide more targeted content and ads to users interacting with specific photos, a win-win. As metadata is used for end-user consumption in these cases, the unique challenge of really needing to support multilingual tagging and descriptions arises, with its own set of challenges.

With companies like Adobe, IBM, Google, and Microsoft pouring a ton of resources into machine learning, expect a lot of changes and improvements in the coming years. Relatively soon, computers will outperform humans in classification and analysis.

As it relates to Digital Asset Management, it remains to be seen precisely what the exact improvements will be. But one thing is certain: Machine learning technology promises a lot of exciting possibilities.

written by: Martin Jacobs (GVP, Technology)

How Machine Learning Can Transform Digital Asset Management

As the use and need for digital assets increase, so too does the cost and complexity of Digital asset management (DAM) — especially in a world where people are adopting devices with screens of all sizes (e.g., desktop, mobile, tablet, etc.).

DAM, however, is a challenge for many organizations. It still involves frequent manual labor, but machine learning is starting to change that.

Machine learning has already given us self-driving cars, speech recognition, effective web searches, and many other benefits over the past decade. But the technology can also play a role in classifying, categorizing, and managing assets in the years to come.

Machine learning can support DAM in areas such as face recognition, image classification, text detection, people recognition, and color analysis, among others. Google PlaNet, for example, can figure out where a photo was taken based on details embedded in it. Google Photos is using it to improve the search experience. Machine learning has already taken a role in image spam detection. Taken together, this all points to the need for DAM tools to start incorporating advanced machine-learning capabilities.

A Practical Test

Recently, Google released its Cloud Vision API. The Google Cloud Vision API enables developers to understand the content of an image by encapsulating powerful machine-learning models in an easy-to-use REST API. It quickly classifies images into thousands of categories (e.g., “sailboat”, “lion”, “Eiffel Tower”, etc.). It detects individual objects and faces within imagines. And it finds and reads printed words contained within images.

For Razorfish, this was a good reason to explore using the Vision API together with a DAM solution, Adobe AEM DAM. The result of the integration can be found on github.

Results screenshot

We leveraged text-detection capabilities, automation classification techniques, and the landmark detection functionality within Google’s API to automatically tag and assign other metadata to assets.

Benefits and Setbacks

Integrating the Vision API provided immediate benefits:

  • Automated text detection can help in extracting text from images, making them easily accessible through search.
  • Automated landmark detection helps in ensuring that the appropriate tags are set on digital assets.
  • Auto-classification can support browse scenarios for finding the right assets.

But there were also some shortcomings. For example, an image of a businesswoman in a white dress was identified as a bride. In other instances, the labels were vague or irrelevant. Though inconvenient, we expect these shortcomings to improve over time as the API improves.

Even with these drawbacks unaddressed, automated detection is still very valuable — particularly in a DAM scenario. Assigning metadata and tags to assets is usually a challenge, and automated tagging can address that. And since tags are used primarily in the authoring environment, false classifications can be manually ignored while appropriate classifications can help surface assets much broader.

The Evolution of DAM Systems

One frequent point in implementing DAM systems is asset migration. I have seen many clients with gigabytes of assets wonder whether to go through the tremendous effort of manually assigning metadata to them.

There’s a quick fix: Auto-classification techniques using machine learning will improve and speed up this process tremendously.

With the benefits around management and migration, machine learning and other intelligence tools will therefore start becoming a key component of DAM systems — similar to how machine learning is already impacting other areas.

Lastly, incorporating machine learning capabilities in DAM solutions will also have architectural implications. Machine intelligence functionality often uses a services-based architecture (similar to the APIs provided by Google) as it requires a significant or complex set of compute resources. As DAM systems start to incorporate them at its core, it will be more difficult for those solutions to support a classic on-premises approach — causing more and more solutions to migrate to a hosted software as-a-service (SaaS) model.

Bottom line? Consider incorporating machine learning into your DAM strategy now, and look at how it can be applied to your digital asset management process.

written by: Martin Jacobs (GVP, Technology)

Diffusing Automation Arguments: The Inevitability of Automation

As mentioned in one of my previous posts, delivering a successful Cloud architecture necessitates the use of automation. Unfortunately, replacing manual tasks with code takes effort, and is therefore not always used. Here are some key arguments against the adoption of automation:

Priority

“We are already on a tight deadline with all the application features that need to be incorporated.”

Automation is critical to the success and longevity of your product. What’s also true, though, is that this is an industry of tight deadlines, stretch goals, and additional features. You might wonder if you have time to automate.

In this case, unit testing is an interesting comparable situation. Often times, unit testing hasn’t always taken priority in the application development process due to time constraints. It has been put off until the end of development phase with a secondary status. However, unit testing has slowly received the priority it deserves, as it has become clear it provides the benefits in the long run.

And as much as testing is important, automation is even more critical. Automation is an actual part of your runtime application, and should be treated at the same level as your code. The features and capabilities for automation should therefore be included in the application/solution backlog and should be given the same treatment as other features and functionality.

Skills

“We don’t have the skills in-house. Even if we were to use a vendor, we wouldn’t be able to maintain it.”

No doubt, automation is a serious challenge. Automation requires a fundamental shift in mindset for organizations around the need to develop these skills. You may remember that in the early days of web development, it took quite some time for front-end development to become a respected and critical role as say database administration. The automation architect will face a similarly arduous battle for the coming years. For any organization that leverages the Cloud and maintains their own technology platforms, it is a critical role that must be filled or grown within the organization.

Time

“It is faster to do it without automation.”

This is often true for the initial setup. However, considering how quickly Cloud architecture continues to evolve, the time gained from a hasty initial setup could quickly be lost in subsequent change management.

With Cloud architectures incorporating more distinct elements, ensuring consistency across environments is virtually impossible without automation. As a result, without automation, the likelihood of generating defects due to environment mismatches increases quickly when your Cloud architecture grows.

Technologies in Use

“The application technologies we use don’t support automation”

As you architect your application, you identify critical non-functional requirements. For example, security and performance are always part of the decision criteria for the overall architecture stack, and if the technologies selected cannot support the level of performance required, you would evaluate alternative options and select and migrate your architecture to the new solution.

The same applies for automation. If automation cannot be supported with the existing technologies, it is necessary to look at alternatives, and evolve your architecture.

Overwhelming Choices

“We are confused by the technology landscape.”

The amount of solutions in the marketplace can certainly feel paralyzing. There’s Ansible, Chef, and PuppetLabs. There are provisioning tools such as AWS Cloud Formation, Heat, Terraform, and Cloudify. Solutions are constantly evolving, and new vendors are always showing up.

It is difficult to make the right choice of technologies. The selection should be made with the same mindset as selecting the enterprise set of programming languages. It requires an evaluation of which is best suited for the organization. Additionally, a combination of these technologies might be the right solution as well. As you embark on applying automation, here are some tips for being successful:

  • Select a set of automation technologies and stick with it. There will always be pressure to explore alternatives, especially with a quickly changing vendor landscape, but it is important to fully understand your selected technologies before looking at alternatives.
  • Start simple. Amazon Elastic Beanstalk or Heroku are great ways to begin to incorporate automation into your application development workflow and understand how it can further drive productivity and quality.
  • Avoid the framework syndrome and focus primarily on building the automation that is needed for your application. Don’t try to build a framework for automation in the enterprise. The landscape is constantly evolving and frameworks quickly become outdated and superseded.

written by: Martin Jacobs (GVP, Technology)

The Cloud and the 100% Automation Rule

Automation and the Cloud go hand-in-hand. Without automation, the Cloud is just classic deployment with rented servers, instead of your own. You’ll need automation if you want to successfully deliver in the Cloud. This was the case early on in the Cloud era, and becomes even more important now.

As Cloud environments evolve and extend, Cloud architectures consist of far more distinct elements than a standarddedicated architecture. With the emergence of new tools like AWS Lambda, which allows you to run code without provisioning servers, these distinctions are becoming even more pronounced.

As we know, manual tasks are tricky. It can be challenging to consistently perform manual tasks correctly due to quickly changing technology and human error. For that reason, 100% automation becomes an important objective. Any deviation from full automation will create additional challenges.

For example, AWS Cloud hosting quickly becomes complex as organizations struggle to choose between many different instance types. You might not know whether you’d be better off using M3, M4 or C3.

Each decision has its own cost implications. Unless you have achieved the 100% automation target, you are often locked into an instance type due to the difficulties and risks of switching to another one, eliminating an opportunity to benefit from getting the optimal cost/performance benefit.

Our automation tools have greatly improved but we still have work to do. Unfortunately, 100% automation is not always possible. Frequently, manual steps are still required. When you do so, ensure that the manual process is automated as much as possible. I’ll highlight it with a couple of examples.

Provisioning

Many tools automate the setup process for provisioning development, test, and production environments.From Cloudformation to Ansible, Chef, and Puppet, many steps can be automated, and as a result are traceable and reproducible. That said, it would be nice to automate the updates to the provisioning stack further.

To start, the provisioning stack is often a static representation of an ideal architecture. But we live in a fast-paced world, and business moves quickly. Making automation work in dynamic environments can be tricky, particularly when infrastructure needs change, new capabilities are launched, or pricing needs to be optimized. Once your largely static architecture is in place, it is hard to keep it evolving to take advantage of new capabilities.

AWS launched a NAT gateway offering recently, eliminating the need for a NAT instance. For the majority of AWS customers, switching to a NAT gateway will improve the reliability of the overall architecture. Unfortunately, it can be difficult to ensure that this switch happens pro-actively.

I would recommend a scheduled review of new provider capabilities for inclusion. If something is needed, a high priority ticket is submitted to ensure that these new capabilities are incorporated with the same priority as code enhancements or defects. If necessary, the provisioning of new environments can be blocked until these tickets are addressed.

Management

Tools that automate environment management also exist. Many Cloud environments can deploy patches and upgrades automatically.

However, commercial or open source products are often deployed in these Cloud environments, and many don’t have the tools to automate the communication of new releases, patches or other updates. Checking for updates becomes a manual process.

To automate the manual process, use a tool like versionista.com to check whether a vendor page lists hotfixes and release updates changes. Similar to the provisioning scenario, if a change gets detected, create a ticket automatically with the right priority, ensuring its implementation.

Optimization

We will start to see real savings once we optimize Cloud infrastructure. However, once the architecture is in place it is challenging to optimize further. This must be a critical core capability for any technology team.

We can optimize development and test environments. Often neglected after a system has launched, we have managed to eliminate manual processes by implementing an automatic shutdown of instances after low usage. The DNS entry for the instance is redirected to the continuous integration environment, allowing testers or developers with the right privileges to restart the instance.

We can also improve upon cost management. A common approach for disaster recovery is to copy data snapshots to another region. However, as the site evolves the data size increases and the disaster recovery process becomes more expensive. How do you track when you should re-architect the process?

Cost management tools like Amazon Cost Explorer focus on products (e.g. EC2, bandwidth), not processes or features. To ensure optimal cost management, you should automatically map the cost data mapped to your processes using tags. Enforce the existence of tags through automated checking, and also automate the processing of the report. This will provide the team with clear indications on where to invest in optimization.

Challenges in Automation

Automation, like anything else, has its challenges. For a Cloud-optimized environment, it is critical to reach for the 100%. If you cannot achieve that, automate the necessary manual processes 100%.

written by: Martin Jacobs (GVP, Technology)

"The cloud is the new normal" – Highlights from AWS re:Invent 2015

reinvent

AWS re:Invent took place in Las Vegas the first week of October, and it has become one of the premier events in the technology industry, with close to 20,000 attendees and numerous exhibitors.

The conference was kicked off with a keynote from Andy Jassy, senior vice president of Amazon Web Services, who declared “The cloud is the new normal” to his audience. Starting off with presenting some impressive growth numbers, the announcements made in his sessions were focused on getting more enterprise customers to migrate to AWS, for example by simplifying the tasks of collecting and analysing data, streaming data, moving large amount data to cloud and migrating existing databases to different database management systems in the cloud.

The second keynote, presented by Amazon Web Services CTO Werner Vogels, was more focused on new development tools, as well as a new offering around the Internet of Things (IoT).

Here are the most significant announcements made at Re:Invent in various area: Database During the conference Jassy took aim at Oracle, the “Old Guard”, which is currently the biggest provider of traditional databases. “It’s rare that I meet an enterprise that isn’t looking to flee from their current database provider,” he said.

AWS strengthened their database offering by adding new features and compatibilities to the Relational Database Service (RDS). While announcing MariaDB as a fully-managed service on RDS it also introduced set of tools to make database migration to cloud simpler.

aws_schema_conversion_tool

MariaDB is a fork of MySQL and is targeted towards developers who are running LAMP applications but are looking for an alternative to MySQL. AWS Database Migration Service intend to help enterprises to migrate their databases to low cost AWS RDS alternatives with minimal downtime. It supports all widely used database platforms, and performs schema and code conversion for migrations between database engines.

Analytics

Company added another service to their enterprise portfolio, Quick Sight. It is a business intelligence tool, which has been developed to compete with IBM’s Congos, Microsoft’s Power BI, etc. Quick Sight makes it easy to build visualisations, perform ad-hoc analysis, and quickly get business insights from data and designed to be easy for non-technical people to use.

Amazon also launched Amazon Kinesis Firehose which is the easy way to load streaming data into AWS. It can capture and automatically load streaming data into Amazon S3 and Amazon Redshift, enabling near real-time analytics with existing business intelligence tools. It is a fully managed service.

IOT

AWS IoT is one of the major announcement from Amazon. It’s a managed cloud platform which will allow customers to connect and manage billions of devices and enable them to process, analyse and act on the data. It leverages the MQTT protocol, is integrated with a large number of IoT devices already, and allows for a rules based management of these endpoints. With a built-in shadow state mechanism, it makes interacting with occasionally connected devices much simpler.

IoT

Mobile

AWS Mobile Hub is a mobile back-end as a service and which allows iOS and Android developers to easily add commonly used features including user authentication, data storage, backend logic, push notifications, content delivery, and analytics.

Security

For most enterprises security is a primary concerns when moving to a cloud infrastructure. AWS announced new services like Web Application Firewall (WAF) and AWS Inspector to help administrators boost security of their infrastructure.

AWS WAF is a web application firewall that helps protect web applications from attacks by allowing Web Administrators to configure rules that allow, block, or monitor (count) web requests based on conditions that you define.

Amazon Inspector is an automated security assessment service that helps minimize the likelihood of introducing security or compliance issues when deploying applications on AWS. Amazon Inspector automatically assesses applications for vulnerabilities or deviations from best practices. After performing an assessment, it produces a detailed report with prioritized steps for remediation.

Compute

AWS Lambda was launched at re:Invent last year and it has quickly become one of most popular services on AWS. Lambda allows to easily build server-less systems that need no administration and can scale to handle a very large number of requests. This year AWS made some significant enhancement by allowing to develop AWS Lambda function code using Python, maintain multiple versions of function code, invoke code on a regular schedule, run functions for up to five minutes and, VPC support.

Data Transfer

AWS unveiled a new PC-Tower-sized storage appliance called “Snowball” which makes easy for large enterprise customers to transfer petabyte’s of data into the cloud. Snowball addresses common challenges with large-scale data transfers including high network costs, long transfer times, and security concerns. Each Snowball device can hold up to 50 terabytes and can be shipped directly to the company for uploading to AWS servers.

snowball

Containers

AWS announced the AWS EC2 Container Registry, allowing you to manage all your container resources on AWS, and manage access through IAM.

containers

Conclusion

Maintaining growth on a scale like AWS is difficult, but with the announcements made at reinvent, AWS is well positioned to gaining further adoption with enterprise customers.

In addition, AWS talks about re:Invent as an educational event, and they were very successful in achieving this in 2015. The sessions we attended were of a very high quality. The sessions are also posted online at https://www.youtube.com/user/AmazonWebServices/featured and http://www.slideshare.net/AmazonWebServices/tag/reinvent2015

Highlights from the 2015 Razorfish Tech Summit

ray-and-andy-web

Today marked the seventh iteration of the Razorfish Tech Summit, which brought together more than 200 technology industry executives in New York City to discuss software innovation. The two-day, invitation-only event was collaboratively sponsored and created with Adobe, Amazon Web Services, Appirio, Dynatrace, hybris, IBM, Rackspace, and Yahoo.

The Lab at the Tech Summit included hands-on demonstrations, in partnership with NYC Media Lab, which gave attendees the chance to experience technology projects produced by students from schools including Parsons, New York University and Columbia University. The students were tasked with inspiring the next wave of technology and were given the opportunity to demonstrate their projects alongside installations from Razorfish Global, Rosetta, Dynatrace and Hybris. Popular demonstrations included:

GlassMouse, which invigorates streetscape and mallscape windows with new invisible powers of interactivity; Flatline, a visualization of an individual’s social media presence as an electrocardiograph; and Pure, a speculative wearable piece commenting on the growing issue of pollution and functionality of clothing. This project is a response to the lack of concern for air quality and the impracticality of the modern fashion industry.

Along with the interactive demonstration area, a lineup of panel discussions inspired attendees throughout the day. The event kicked off with an opening talk from Razorfish Global Chief Technology Officer Ray Velez, who encouraged executives to “build experiences that are timely, relevant and smart.” Velez mentioned Marc Andreessen’s quote “software is eating the world” to highlight the ways in which technology is creating an environment that is bringing together physical and digital experiences. Despite the changes in technology, Velez urged: “Consumers are more in charge than ever before. They are expecting services that are built just for them.” Velez emphasized how machine learning, cloud, and big data technology are critical in putting the customer at the center of their business endeavors.

The consumer experience was at the heart of the first presentation of the morning led by Piers Fawkes, who is the founder and editor-in-chief of PSFK.com and PSFK Labs. Fawkes gave the audience a preview of his “Future of Retail” report, noting the innovative ways retailers and brands are using technology to connect with consumers. He explained that the new shopping experience revolves around ten pillars, ranging from creating confidence among shoppers to delivering a delightful, unique retail experience.

Next, Dr. Steven Abrams, the director of Watson Ecosystem Technology at IBM, took the stage to explore cognitive computing and how humans and machines are collaborating in surprising ways. “It is not man versus machine now,” Dr. Abrams said. “It is man working together with machine.”

He cited various examples of how cognitive computing might work in different industries, whether it is a chef using a collaborative partner for recipe ideas or a physician working to better diagnose patients by analyzing the medical records of those who experienced similar symptoms. A video shown during the presentation also showcased the work of Elemental Path in New York City, which is using IBM’s Watson technology to develop connected toys to aid children in learning.

Following Dr. Abrams’ presentation, Razorfish leaders Christopher Follett (Executive Creative Director) and Eric Campdoras (Creative Technology Director) encouraged the audience to view the retail environment in a new way. Their demonstration of “Talk & Shop,” which involves a mix of iterating prototyping, artificial intelligence, synchronization and voice capabilities, showcased how mobile can enhance a consumer’s shopping experience.

And as today’s consumer is very much influenced by mobile, David Iudica, Yahoo’s director of strategic insights and research, revealed findings from a global study conducted with Flurry Analytics with 6,000 smartphone users.

Highlights of the study include:

  • People are adopting smartphones quickly due to the multiple products it has replaced

  • More efficiency and a better experience will lead to increased smartphone usage

  • Phablets are the fastest growing device; as screen sizes grow, engagement will increase

  • In the U.S., people spend three hours and 40 minutes a day on mobile devices

  • Native mobile advertisements on premium earn three times more attention from consumers than the static banner ad

Following this deep dive into smartphone usage, Andrew McAfee, principal research scientist of MIT Sloan School of Management, shared how machines and humans can work together, stressing people still have the upper hand. “Our brains and our minds clearly do remain valuable even in a world of advanced technology,” McAfee said. We all felt reassured when he guided us to a recent report, “The Growing Importance of Social Skills in the Labor Market.” He also encouraged the audience to “be more geeky” and to embrace data to make better decisions. The team appreciated hearing the geeks (data-driven decisions) vs. HiPPOs (highest paid person’s opinion) debate.

Amyx+McKinsey Founder Scott Amyx, Founder & CEO then discussed how wearables and the Internet of Things can provide new capabilities that impact the brand engagement cycle. He shared examples from retail and omni-channel marketing that showcase how to engage consumers in new ways. He also explained how wearables and the Internet of Things can enable better experiences by measuring emotion. One statistic he shared that surprised the crowd was 55% of what we as people communicate is done via body language.

After Amyx’s talk, David Nuescheler, the VP of Enterprise Technology at Adobe Systems Incorporated, delivered a presentation entitled “Technology Paradigm Shifts.” He explained why Adobe’s priorities in open, cloud, and mobile experiences are critical to their future success. He wrapped up with the new truism, “the only constant now is the acceleration of change.” RAZORFISH_SPEAKERS_1883

Next, NYC Media Lab Executive Director Justin Hendrix led a panel with the Razorfish Emerging Experiences team including, Steve Dawson, Luke Hamilton, Charles Fletcher and Kat McCluskey. The group shared about the future of augmented and virtual reality. “These technologies are beginning to create a lot of buzz,” Hendrix said, noting that $150 billion will be invested in the space by 2020. Sensor technology and 3-D design were cited as innovations in the category.

RAZORFISH_SPEAKERS_2360

The discussion then transitioned to the NEXUS Customer Intelligence Marketing Operating System (MOS). Panelists Samih Fadli, Chief Intelligence Officer, Razorfish Global; Dmitri Tchikatilov, Global Business Development Executive for Digital Advertising at Amazon Web Services; and Tom Kotlarek, Executive Vice President, Technology Razorfish Global, shared the ways data can be leveraged to drive targeted customer experiences and boost engagement. With incredibly powerful big data technology, enabled through cloud computing, Nexus has a great answer to the oncoming deluge of data coming from the greater and greater proliferation of data. Nexus has a vision to use data to listen to customers enabling breakthrough experiences.

Visa’s Senior Vice President Global Head of Digital & Marketing Transformation Shiv Singh—our former Razorfish colleague— delivered a dynamic presentation, “Five Ideas for Marketing Transformation.” Singh says this kind of marketing revolves around open source brand building, transmedia storytelling, ubiquitous engagement, being anchored in owned media experiences that mirror the model of a publisher or magazine, and adopting a marketing checklist that takes full advantage of digital. “Software is becoming culture,” Singh said. “Technology is culture—it is life.”

Hilary Mason, the founder and CEO of Fast Forward Labs, which advises brands on data strategy, closed the Tech Summit with a look at data science. Mason presented on “Innovation Through Data,” exploring how companies like Reddit, Foursquare and Google Maps are utilizing data effectively. She mentioned that companies of this nature are driven by data due to access to sophisticated computation, the ability to collect and store it, and being adept at analyzing it in an efficient manner. Mason also touched on machine learning. “We are at the beginning of machine learning applications,” she said. “We will see machines that interface with us.” Check out her recent book, Data Driven: Creating a Data Culture, on how to use data to help drive your business.

Hands down, it was our best Tech Summit yet. Watch this space—in the coming days we will publish another post that contains links to many of the presentations.