FAQ on Rapid Development Environments - RDEs in AEM as Cloud Service

As an AEM Developer, we always wanted a faster deployment process enabled in AEM as Cloud Service Environments. There is a good news for all of us!

Why we need faster deployments?

Being a curious developer, one of the newly developed feature we wanted to test quickly but we know usually a deployment takes an average of 40 minutes in AEM cloud service. The average time for Adobe Experience Manager (AEM) cloud deployment can vary depending on several factors, such as the complexity of the implementation, the size of the content repository, the number of customizations required, and the chosen cloud deployment model.

Why the deployments take time on AEM as Cloud Service Environments and what is the relevance of a Rapid Development Environments (RDEs)?

Usually a code deployment on AEM as cloud service goes through a set of code security and quality rules which makes it a slow process. Assume a new feature has to be tested quickly on AEM as cloud which is already working on AEM as cloud local sdk. Using RDE, a developer can quickly deploy the changes and test it. And successful RDE tested code can be deployed through normal AEM as cloud deployment pipelines.

What is the new feature ‘Rapid Deployments’ in AEM as cloud service which enables quick deployments?
 
Rapid deployments in Adobe Experience Manager (AEM) as a cloud service refers to the ability to quickly and efficiently deploy AEM instances in the cloud. Rapid deployment is achieved through a combination of pre-configured environments, CLi plugins, and configurations.

RDE provides below advantages 

-RDE enables a faster development cycles
-Efficient Testing on a ‘near to prod’ environment which gives confidence to developers
-Enables collaboration between developers to test related features by increasing productivity
-Avoids the long waiting deployment process

By default RDE gets set to the most recent AEM version.

How developer works on RDEs

The developer uses command line tools to invoke the RDEs on cloud.

Once the user is done with a feature testing and a successful local build completion, he can use the CLI commands to invoke the RDE deployment. If multiple developers want to use RDEs, they will have to commit the code in feature branch /club the code and use CLI to test the feature at a time.

How do we work with RDEs?

Please ensure the developer is having latest version of Adobe I/O Runtime Extensible CLI . Steps given below,

Step1

$ npm install -g @adobe/aio-cli - To install the Adobe I/O Runtime Extensible CLI,
$ aio plugins:install @adobe/aio-cli-plugin-cloudmanager - To Install Cloud Manager plugin
$ aio plugins:install @adobe/aio-cli-plugin-aem-rde - To install AEM RDE plugin
$ aio aem:rde:status - To verify the AEM RDE plugin installation and configuration 

Step 2

Once basic installation is done, we need to configure few things,
$ aio config:set cloudmanager_orgid <org-id>  - Set the organization id of RDE
$ aio config:set cloudmanager_programid <program-id> - Set the program id of RDE
$ aio config:set cloudmanager_environmentid <env-id> - Set the environment id of RDE

Step 3

Validate the configurations now using below command,
$ aio config:list - To verify the current config values

Step 4 

Once above steps are completed, do a code build on developer machine using below command
 mvn clean install -PautoInstallSinglePackage

Step 5
 

After successful code build, developer can use CLI commands to push the code/ packages to RDE environment.

Note: If one developer wants to use the RDE for new feature testing, he will have to ensure the RDE is ‘reset’ before new deployments.

A pictorial representation of the RDE development flow is given below.

What all can be deployed using RDE approach

- [+] AEM code & content package (all, ui.apps)
- [+] OSGi bundle and config files
- [+] Any individual files like HTL, .content.xml (dialog XML)
- [+] Apache and Dispatcher configs deployment as a zip file

Is this(RDE) feature available for all readily?
This is an on-demand feature at present, we will have to contact Adobe for enabling this feature. But every licensed customer is expected to get an RDE each in couple of weeks from now and additional RDE is license based.

How do we enable it once RDEs are provisioned?
In the ‘Environments’ > Add Environment section, we have a new options called ‘Rapid Deployment’. Once you provide necessary details, we can see the new rapid deployment added under environment section.

Will RDE work for sandbox programs?
Yes, RDE works for sandbox programs as well as production programs

Few useful commands for RDEs (assuming we have a ‘sample aem wknd guides project’)

Install the 'all' package

• aio aem:rde:install all/target/aem-guides-wknd.all-2.1.0-SNAPSHOT.zip

To deploy the OSGi bundle,

• aio aem:rde:install target/aem-guides-wknd.core-2.1.0-SNAPSHOT.jar

OSGi configuration (Deploy individual config file)

• aio aem:rde:install ui.config/src/main/content/jcr_root/apps/wknd/osgiconfig/config/org.apache.sling.commons.log.LogManager.factory.config~wknd.cfg.json

Deploy the complete config package

• aio aem:rde:install target/aem-guides-wknd.ui.config-2.1.0-SNAPSHOT.zip

Install the 'dispatcher' zip

• aio aem:rde:install dispatcher/target/aem-guides-wknd.dispatcher.cloud-2.1.0-SNAPSHOT.zip

Install component Dialog and related HTL files.

• aio aem:rde:install target/aem-guides-wknd.ui.apps-2.1.0-SNAPSHOT.zip

Individual HTL file

• aio aem:rde:install ui.apps/src/main/content/jcr_root/apps/wknd/components/helloworld/helloworld.html -t content-file -p /apps/wknd/components/helloworld/helloworld.html

where p is the path and t is the type.

Install a dialogue XML

• aio aem:rde:install ui.apps/src/main/content/jcr_root/apps/wknd/components/helloworld/_cq_dialog/.content.xml -t content-xml -p /apps/wknd/components/helloworld/_cq_dialog/.content.xml

To list out all commands

• aio aem:rde:install --help

There are additional RDE commands -

• aio aem rde delete -  Delete bundles and configs from the current rde.
• aio aem rde history - Get a list of the updates done to the current rde.
• aio aem rde install  - Install/update bundles, configs, and content-packages.
• aio aem rde reset   - Reset the RDE
• aio aem rde restart  - Restart the author and publish of an RDE
• aio aem rde status - Get a list of the bundles and configs deployed to the current rde.
• aio where - To switch or know which organization you are currently logged in to

Summary:

Using RDE, a developer can test his features quickly and efficiently without waiting for a build & deployment pipeline.

AEM As Cloud Service Video Series

How to update JDK version of Cloud Manager within your program as part of the build

 AEM AS CLOUD SERVICE VIDEO TUTORIAL SERIES

What are the available JDK versions with cloud manager?

Java versions installed are Oracle JDK 8u202 and 11.0.2.

Some times, when we deploy our code, people complaint java version is not matching (for e.g. we are using a JDK 11 specific API and this Java class is giving error in cloud manager) We will see how to set the relevant java version now.

How to update the Java version of project build environment in AEM As Cloud Manager

By default, projects are built using Java 8. Customers who want to use Java 11 in their projects can do this using the Apache Maven Toolchains Plugin.

In the pom.xml file, add a <plugin> entry that looks like this:

<!-- START -->
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-toolchains-plugin</artifactId>
            <version>1.1</version>
            <executions>
                <execution>
                    <goals>
                        <goal>toolchain</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <toolchains>
                    <jdk>
                        <version>11</version>
                        <vendor>oracle</vendor>
                    </jdk>
                </toolchains>
            </configuration>
        </plugin>
<!-- END -->

A few FAQs about AEM As Cloud Service build environment are given below

What is the OS used for AEM as CLoud service?

Build environment is Linux-based, derived from Ubuntu 18.04.

At present what is the maven version?

The installed maven is - Apache Maven 3.6.0 version.

Which java versions are supported by AEM as cloud service?
Java versions installed are Oracle JDK 8u202 and 11.0.2.

Which repository is configured as part of Maven Build ?
adobe-public

Any other tools or packages available in AEM As Cloud environment?
unzip, bzip2, libpng, imagemagick, graphicsmagick

My code has some script with Python or ruby script and I need to trigger it as part of my code build. Is this possible?

For this we need an appropriate language interpreter installed. This can be done by calling the 'exec-maven-plugin' to invoke APT. See an example below,

<!-- START -->
        <profile>
            <id>install-Ruby</id>
            <activation>
                <property>
                        <name>env.CM_BUILD</name>
                </property>
            </activation>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.codehaus.mojo</groupId>
                        <artifactId>exec-maven-plugin</artifactId>
                        <version>1.6.0</version>
                        <executions>
                            <execution>
                                <id>apt-get-update</id>
                                <phase>validate</phase>
                                <goals>
                                    <goal>exec</goal>
                                </goals>
                                <configuration>
                                    <executable>apt-get</executable>
                                    <arguments>
                                        <argument>update</argument>
                                    </arguments>
                                </configuration>
                            </execution>
                            <execution>
                                <id>install-ruby</id>
                                <phase>validate</phase>
                                <goals>
                                    <goal>exec</goal>
                                </goals>
                                <configuration>
                                    <executable>apt-get</executable>
                                    <arguments>
                                        <argument>install</argument>
                                        <argument>-y</argument>
                                        <argument>--no-install-recommends</argument>
                                        <argument>ruby</argument>
                                    </arguments>
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>
        </profile>
<!-- END -->

Site Template in AEM FAQ

 Site Template in AEM FAQ

What is Site template (known as Sites 30)?

Site template is a predefined template which can be the starting point of any website creation. It contains basic theming, page templates, configurations and sample content. Instead of going to the archetype way of creating step by step approach of developing a website, Site template helps to utilize the out of the box components and templates and setup a website in shorter period of time. Once the site is up, the developers parallely work towards achieving the brand styles and customization (utilizing CSS, JS)

This is the new recommended Adobe way of developing an AEM site

Can I get it installed on my local AEM instance?
As of now the Site Template is enabled as part of AEM as A Cloud Service ONLY. We will have to contact Adobe to get this feature enabled in our AEM AS CLOUD.

Where I can get the templates?
This is an open-sourced project like Core Components. It can be downloaded from Basic Site Template project on GitHub, but always ensure you use the latest release.
https://github.com/adobe/aem-site-template-basic

What is themes in Sites?

Themes used to depict styles. We can modify theme sources of an Adobe Experience Manager Site to apply brand specific styles.

Where I can find more details?

We can always get demo and tutorials on below link.
https://experienceleague.adobe.com/docs/experience-manager-learn/getting-started-wknd-tutorial-develop/site-template/overview.html?lang=en

FAQ on AEM as a cloud service

1) Which are all the environments available as part of AEM as a cloud service?

Four types of environments available with AEM as a Cloud Service:

• Production environment - for the business practitioners
• Stage environment : performance and quality tests before changes to the application are pushed to the production
• Development environment -  developers to implement AEM applications
• Demonstration environment : Training , demos, pocs etc - is simplified to a single author node, all others having min 2 author nodes

2) Which are all the types of programs available as part of AEM as a cloud service?

Two types of programs are initially available for AEM as a Cloud Service:

• AEM Cloud Sites Service
• AEM Cloud Assets Service

3) What we get as part of AEM as a cloud license?

When we get the license we will have

• Code repository (Git)  1
• Baseline image (Sites or Assets)  1
• Stage and production environment set (1:1) 0 or 1
• Non-production environments (development or demonstration) 0 to N
• Pipeline for each environment 0 or 1

**Note:* The author tier will contain all Sites and Assets functionality for all programs, but the Assets programs will not have a publish tier by default

4) What are all the changes in Author, Publish & Replication in AEM as a Cloud Service

Author and publish features:

Both author and publish tiers always accessed via a load balancer. But publish tier, a Continuous Delivery Network (CDN) is always available.

The method of author to publish replication has upgraded now. AEM as a Cloud Service Sling Content Distribution which  allows one to distribute Sling resources between different Sling instances.{The API works at path level and the distribution agents basically enable distribution of specific paths between instances.}This uses a pipeline service run on Adobe I/O, which is external to AEM.

Replication changes - The replication agents used in previous versions of AEM are no longer used or provided, which might impact the following areas of existing AEM projects:

• Custom workflows that push content to replication agents of preview servers for example.
• Customization to replication agents to transform content
• Using Reverse Replication to bring content from publish back to author

5) Do we have all types of runmodes as in previous version of AEM?

The answer is NO.

Run modes that are defined typically include the service (author and publish) and the environment (dev, stage, prod).

Pattern
<service>.<environment_type>

Fr eg: author.dev or publish.prod

The supported runmode configurations in AEM as a cloud service are:

• config ( The default, applies to all AEM Services )
• config.author ( Applies to all AEM Author service )
• config.author.dev ( Applies to AEM Dev Author service )
• config.author.stage ( Applies to AEM Staging Author service )
• config.author.prod ( Applies to AEM Production Author service )
• config.publish ( Applies to AEM Publish service )
• config.publish.dev ( Applies to AEM Dev Publish service )
• config.publish.stage ( Applies to AEM Staging Publish service )
• config.publish.prod ( Applies to AEM Production Publish service )
• config.dev (*Applies to AEM Dev services)
• config.stage (*Applies to AEM Staging services)
• config.prod (*Applies to AEM Production services)

** OSGI configuration that has the most matching runmodes is used.

6) How do we do local development when AEM is in cloud?

For local development:
The following artifacts are made available to the developers:

• The AEM as a Cloud Service QuickStart: a .jar based, standalone installer of the latest AEM code base, with the same functional and API surface.
• The AEM as a Cloud Service Dispatcher SDK: an image-based process for testing and validating Dispatcher configurations locally

[* The quickstart is a simple author environment where the majority of the extensions can be developed and tested - does not allow for all AEM Sites and AEM Assets functionalities]

7. What are all some of the important terminologies which we should be aware of?

New important terminologies w.r.t. cloud

• AEM as a Cloud Service - The cloud-native way of leveraging the AEM applications
• AEM Image - A deployable artifact that contains the AEM product code together with the customer code.
• Golden Master - The AEM publish tier.
• Orchestration Engine - AEM as a Cloud Service uses an orchestration engine to ensure that all author and publish services are scaling as and when needed.
• Asset microservices - Cloud-based digital asset processing services that cater to various asset processing use cases, such as rendition generation, PDF processions, subasset handling, text extraction etc.

8) What are all the types of users and roles available in AEM as a cloud?

Users & Roles

Cloud Manager currently defines four roles for users which govern the availability of specific features:

• Business Owner - defining KPIs, approving production deployments
• Program Manager -  team setup, review status
• Deployment Manager - execute stage/production deployments
• Developer - Develops and tests custom application code , do git operations

* There is a role 'Content Author' who does not interact with Cloud Manager. May use Cloud Manager Program Switcher (having navigated from Experience Cloud) to access AEM

9) How do we upgrade from existing AEM version to Cloud Service

> Planning
Access cloud service readiness - determine areas that will require refactoring to be compatible with AEM as a Cloud Service.(Source code Vs deprecated features, New path structure w.r.t AEM as cloud). Then estimate the plan.
Review resource planning - identify resources, create a team, and map out roles and responsibilities
Establish KPIs - Define KPIs to help your team focus on what matters the most.

> Execution
- onboad -familiarize & deploy the code to cloud service
- Integrate - the Git and deploy the code, content transfer, code refactor( Use tools where ever possible For eg: Asset Workflow Migration Tool, Dispatcher Converter, Modernization Tools. )
- Configure - user roles and other things on Admin console of cloud manager

> Post Go-Love
clean-up of temporary files,
review best practices for continuous development
manage logs

10) What are all the options available to troubleshoot anything in AEM as a cloud service?

The following tools are available to troubleshoot AEM as a Cloud Service environments:

• Developer Console
• CRX/DE Lite
• Managing Logs

11)  What does the SDK For Local Development contains in AEM as a Cloud Service?

SDK is comprised of the following artifacts

• Quickstart Jar - The AEM runtime
• Java API Jar - all allowed Java APIs that can be used to develop against AEM as as Cloud Service(Previously known as Uber Jar)
• Javadoc Jar - The javadocs for the above JAR
• Dispatcher Tools - set of tools used to develop against Dispatcher locally

12) How does the Maintenance tasks became more users friendly now?

With AEM as a Cloud Service, the need for customers to configure the operational properties of maintenance tasks is minimal. Customers can focus their resources on application-level concerns, leaving the infrastructure operations to Adobe.

Below given customer owned the configuration

• Ad-hoc Task Purge
• Workflow Purge
• Project Purge

13) Can you explain mutable vs immutable in AEM as a Cloud Service?

Mutable Vs immutable

• /apps and /libs are considered immutable areas of AEM as they cannot be changed (create, update, delete) after AEM starts (i.e. at runtime). Any attempt to change an immutable area at runtime will fail.
• Everything else in the repository, /content , /conf , /var , /home , /etc , /oak:index , /system , /tmp , etc. are all mutable areas, meaning they can be changed at runtime.
• Oak indexes are mutable at run time
• *  /oak:index configurations are part of the Code Package and not part of the Content Package

All you need to know about AEM Assets as a Cloud Service

The new 'AEM Assets as a cloud service' which is part of AEM as a cloud (platform as a service solution) provides Digital Asset Management capabilities(storage, managing metadata online, versioning, upload and download) with below extended features.

• Based on asset microservices(asset ingestion and processing).
• Smart capabilities, such as AI/ML
• Highly scalable
• Always current
• Always available
• Auto scaled, deployed and monitored

In older AEM all the asset operations happened at AEM Author instance - which consumes considerable CPU, memory, and I/O resource.Asset processing and storage requirements demand resources which in turn create performance issues impact authoring and browsing experience of end users.

A High-level Architecture of Assets as a Cloud Service can be seen below

The generic steps followed in sequence are,

• Clients send an upload request - then start uploading binary directly to cloud
• Once the direct upload is completed, the client notifies AEM
• Now the AEM sends a processing request to Assets Microservice
• The asset microservice now start processing the asset (based on the rendition request from AEM) - asset microservice runs relevant microservices for this. They access the binary from cloud and processed assets are also placed in binary cloud.
• Now assets microservice notifies AEM that renditions are available.

Assets as a Cloud Service Vs AEM Asset upload on premise

Assets as a Cloud Service uses direct binary access principle for upload and download - Previously Assets were uploaded directly to AEM author instance for processing.

Assets as a Cloud Service uses 'asset microservices' for asset processing, which is external to AEM -  But in older AEM versions, all process happened within AEM.

In Assets as a Cloud Service DAM Asset Update Not available [ asset microservices provide a scalable, readily available service that covers most of the default asset processing (renditions, metadata extraction, text extraction for indexing)]. But in older AEM we had DAM Asset Update workflow as default.

Assets as a Cloud Service comes with post-processing workflows which can be used or customizations(where additional processing of assets is required that cannot be achieved using the processing profiles) -In older AEM we had default + customized workflow steps (Even though it looks as an advantage it had used AEM for all processing).

In Assets as a Cloud Service the standard Asset upload interface is the Touch-enabled UI -  In older version Classic UI was available.

In Assets as a Cloud Service only the new upload APIs are supported -The older AEM Assets HTTP API(AEM 6.5), AssetManager Java API, is deprecated now

Advantages of new cloud

• The uploaded binaries do not go through AEM, which is now simply coordinating the upload process with the binary cloud storage configured for the deployment. finally clients get direct access to them to carry out their work. This minimizes the load on networks and duplication of binaries stored.
• Binary cloud storage is fronted by a Content Delivery Network (CDN, Edge Network), which brings the upload endpoint closer to the client, helping to improve upload performance and user experience, especially for distributed teams uploading assets
• More scalable and performant handling of asset uploads.

Ways of uploading Assets to Assets as a Cloud Service
Upload using web interface, Adobe Asset Link, AEM desktop app or custom applications which uses the new HTTP API.

Post-processing workflows
There are cases where we need additional processing to be done, which are not done by asset microservices(For eg. Generating a rendition which requires an integration with other application), additional post-processing workflows can be added to the configuration.

Post-processing workflows, once configured, are automatically executed by AEM after the microservices processing finishes. There is no need to add workflow launchers manually to trigger them.

Some examples for Post Processing workflow use cases are:

• Custom workflow steps to process assets.
• Additional processing done by external services.
• Integrations to add metadata or properties to assets from external systems

How to create Post - Processing Workflows: Steps involved

1. Create one or more workflow models. - they are of regular AEM workflow models
2. Add specific workflow steps to these models.
3. Add 'DAM Update Asset Workflow Completed' Process step at the end(To inform AEM once the processing is done)
4. Create a configuration for the Custom Workflow Runner Service(configuration of an OSGi service) - This ensures the execution of a post-processing workflow model either by a path (folder location) or by a regular expression.

Supported File Formats

Adobe formats - AI, COLLAGE, DN, IDEAS, INDD, INDT, PDF, PROTO, PSB, PSD, XD
Imaging file formats - BMP, EPS, GIF, JPEG, PNG, SVG, TIFF
Image formats in Dynamic Media - PNG, GIF, TIFF, JPEG, BMP, PSD , EPS, PICT
3D formats - DN, gLB, gLTF, OBJ, STL, USDz
Camera Raw file formats - 3FR, ARW, CR2, CR3, CRW, DCR, DNG, ERF, FFF, GPR, IIQ, KDC, MEF, MFW, MOS, MRW, NEF, NRW, ORF, PEF, RAF, RAW, RW2, RWL, SRF, SRW, X3F
Document formats - PDF,DOCX,DOC,PPTX,PPT, XLSX,XLS,ODF,OFG,ODM,ODP,ODS,ODT,EPUB,HTML,PS,RTF,TXT,XML
Document formats in Dynamic Media - AI, PDF, INDD
Video formats - 3G2,3GP,AVI,DIVX,F4V,FLV,M2T,M2TS,M2V,M4V,MKV,MOV,MP4,MPEG,MPG,MTS,OGV,QT,R3D,SWF,WEBM,WMV
Video formats in Dynamic Media for transcoding - MP4,MOV, QT,FLV, F4V,WMV,MPG, VOB, M2V, MP2,M4V,AVI,WebM,OGV, OGG,MXF,MTS,MKV,R3D, RM,RAM, RM,FLAC,MJ2,
Audio formats  - AIF, ASF, M4A, MP3, WAV, and WMA