JCR files to add

After considering the question, all the options I have are:

• Keep logs in memory, save them to CRX every time the user calls info/warn/error
  
  . Pros: Logs are stored in the same location as the metadata of the migration task, easy to find and access. Cons: Potentially slowest and least resource efficient of all approaches when there are a lot of log entries.

• Keep the logs in memory, only store them in JCR at the end of the migration. Pros: Simple refactoring of the current solution, less pressure on CRX during the migration process. Cons: Unable to track progress in real time, potential loss of logs during an unexpected error or instance outage.

• Create a custom type node for each log entry instead of log.txt. Aggregate logs in a text file via a dedicated log servlet. those. /var/migration/uuid/log.txt
  
  or /var/migration/uuid/log.json
  
  . Pros: More JCR way of storing such content. With a custom node, the type and indices must be fast enough to be considered an option. Has variety to support text and json log format. Cons: Fuzzy performance comparison with current approach. Potential problems due to the large number of nodes located at the same level. The user must be aware of the existence of the servlet log, otherwise the user cannot see them in a convenient format. The performance of the log servlet is not clear in the case of a large number of log entries.

• Create log files on the file system (say, in crx-quickstart/logs/migration/<uuid>.log
  
  ), show its contents (if necessary) via API, with the ability to transcribe the response of the log API for 100-1000 lines. Pros: The classic and well-known approach to logging, where log files are stored on the file system. Sling provides custom binding slf4j
  
  to LogBack
  
  all the necessary dependencies LogBack, exported for use in your custom packages. Cons: Separation of logs and task metadata. The user must know the location of the log files on disk.

Starting with option 1 , I realized that the number of log entries could potentially scale to hundreds of thousands - a rare but possible scenario. So finally decided to go with option 4 .

In case anyone comes across a similar task, I am posting the implementation details for parameter 4 here , as it is not as trivial as it might seem at first.

I am using AEM 6.2

(Felix-Jackrabbit-Sling under the hood) and I want each migration task to run - essentially just a separate thread - to create its own log file with a special name - a unique identifier for this migration process.

Now, Sling itself allows you to define multiple log configurations using org.apache.sling.commons.log.LogManager.factory.config

OSGi configuration . However, these log configurations are too simple for this case - you cannot create with it what is called in LogBack a SiftingAppender - a special case of adding a log that will instantiate for individual logs per thread , not once and in the application - in other words, you cannot tell LogBack to create a file on a stream using OSGi config.

So, logically thinking that you want to programmatically use the Sling LogBack configuration at runtime (for example, the moment you downloaded your own kit and activated it), use it to configure such an application for specific loggers. Unfortunately, while there is a lot of documentation on how to configure LogBack through logback.xml

, there are quite a few documents that describe how to do this programmatically using Java LogBack objects such as ch.qos.logback.classic.LoggerContext

, and, like null, explains how to configure this path a SiftingAppender

.

So, after reading sources and LogBack tests, I ended up in this helper class:

import ch.qos.logback.classic.LoggerContext;
import ch.qos.logback.classic.encoder.PatternLayoutEncoder;
import ch.qos.logback.classic.sift.MDCBasedDiscriminator;
import ch.qos.logback.classic.sift.SiftingAppender;
import ch.qos.logback.classic.spi.ILoggingEvent;
import ch.qos.logback.core.Appender;
import ch.qos.logback.core.Context;
import ch.qos.logback.core.FileAppender;
import ch.qos.logback.core.joran.spi.JoranException;
import ch.qos.logback.core.sift.AppenderFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.slf4j.MDC;

import java.util.Objects;

/**
 * This class dynamically adds configuration to AEM LogBack logging implementation behind slf4j.
 * The point is to provide loggers bound to specific task ID and therefore specific log file, so
 * each migration task run will be written in it standalone log file.
 * */
public class LogUtil {

    static {
        LoggerContext rootContext = (LoggerContext) LoggerFactory.getILoggerFactory();

        ch.qos.logback.classic.Logger logger = rootContext.getLogger("migration-logger");

        //since appender lives until AEM instance restarted
        //we are checking if appender had being registered previously 
        //to ensure we won't do it more than once
        if(logger.getAppender("MIGRATION-TASK-SIFT") == null) {
            MDCBasedDiscriminator mdcBasedDiscriminator = new MDCBasedDiscriminator();
            mdcBasedDiscriminator.setContext(rootContext);
            mdcBasedDiscriminator.setKey("taskId");
            mdcBasedDiscriminator.setDefaultValue("no-task-id");
            mdcBasedDiscriminator.start();

            SiftingAppender siftingAppender = new SiftingAppender();
            siftingAppender.setContext(rootContext);
            siftingAppender.setName("MIGRATION-TASK-SIFT");
            siftingAppender.setDiscriminator(mdcBasedDiscriminator);
            siftingAppender.setAppenderFactory(new FileAppenderFactory());
            siftingAppender.start();

            logger.setAdditive(false);
            logger.setLevel(ch.qos.logback.classic.Level.ALL);
            logger.addAppender(siftingAppender);
        }
    }

    public static class FileAppenderFactory implements AppenderFactory<ILoggingEvent> {

        @Override
        public Appender<ILoggingEvent> buildAppender(Context context, String taskId) throws JoranException {
            PatternLayoutEncoder logEncoder = new PatternLayoutEncoder();
            logEncoder.setContext(context);
            logEncoder.setPattern("%-12date{YYYY-MM-dd HH:mm:ss.SSS} %-5level - %msg%n");
            logEncoder.start();

            FileAppender<ILoggingEvent> appender = new FileAppender<>();
            appender.setContext(context);
            appender.setName("migration-log-file");
            appender.setFile("crx-quickstart/logs/migration/task-" + taskId + ".log");
            appender.setEncoder(logEncoder);
            appender.setAppend(true);
            appender.start();

            //need to add cleanup configuration for old logs ?

            return appender;
        }
    }

    private LogUtil(){
    }

    public static Logger getTaskLogger(String taskId) {
        Objects.requireNonNull(taskId);
        MDC.put("taskId", taskId);
        return LoggerFactory.getLogger("migration-logger");
    }

    public static void releaseTaskLogger() {
        MDC.remove("taskId");
    }
}

      

        
        
        
      

    

The part to look at is what SiftingAppender

requires you to implement an interface AppenderFactory

that will create customized log apps for each thread to work with.

You can now get the logger via:

LogUtil.getTaskLogger("some-task-uuid")

      

        
        
        
      

    

And use it to generate log files like the crq-quickstart/logs/migration/task-<taskId>.log

one providedtaskId

According to the docs, you should also release such a registrar as soon as you have done so.

LogUtil.releaseTaskLogger()

      

        
        
        
      

    

And pretty much it is.