Merge branch 'master-remote' into rust

# Conflicts:
#	README.md
#	jplag.cli/pom.xml
#	jplag/src/main/java/de/jplag/options/LanguageOption.java
#	pom.xml

README.md

@@ -48,7 +48,7 @@ Usage: JPlag [ options ] [ <root-dir> ... ] [ -new <new-dir> ... ] [ -old <old-d
 
 named arguments:
   -h, --help       show this help message and exit
-  -l               {java,python3,cpp,csharp,golang,kotlin,rlang,rust,char,text,scheme} Select the language to parse the submissions (default: java)
+  -l               {java,python3,cpp,csharp,golang,kotlin,rlang,rust,scala,char,text,scheme} Select the language to parse the submissions (default: java)
   -bc BC           Path of the directory containing the base code (common framework used in all submissions)
   -v               {quiet,long} Verbosity of the logging (default: quiet)
   -d               Debug parser. Non-parsable files will be stored (default: false)

jplag.cli/pom.xml

@@ -52,6 +52,10 @@
             <groupId>de.jplag</groupId>
             <artifactId>rust</artifactId>
         </dependency>
+        <dependency>
+            <groupId>de.jplag</groupId>
+            <artifactId>scala</artifactId>
+        </dependency>
         <dependency>
             <groupId>de.jplag</groupId>
             <artifactId>scheme</artifactId>
@@ -60,6 +64,10 @@
             <groupId>de.jplag</groupId>
             <artifactId>chars</artifactId>
         </dependency>
+        <dependency>
+            <groupId>de.jplag</groupId>
+            <artifactId>endToEndTesting</artifactId>
+        </dependency>
         <dependency>
             <groupId>net.sourceforge.argparse4j</groupId>
             <artifactId>argparse4j</artifactId>

jplag.endToEndTesting/README.md

@@ -0,0 +1,105 @@
+# JPlag - End To End Testing
+With the help of the end-to-end module, changes to the detection of JPlag are to be tested.
+With the help of elaborated plagiarisms, which have been worked out from suggestions in the literature on the topic of "plagiarism detection and avoidance", a wide range of detectable change can be covered. The selected plagiarisms are the decisive factor here as to whether a change in recognition can be perceived. 
+
+### References
+These elaborations provide basic ideas on how a modification of the plagiarized source code can look like or be adapted.
+These code adaptations refer to a wide range of changes starting from
+adding/removing comments to architectural changes in the deliverables.
+
+The following elaborations were used to be able to create the plagiarisms with the largest coverage:
+- [Mossad: defeating software plagiarism detection](https://dl.acm.org/doi/abs/10.1145/3428206 "Mossad: defeating software plagiarism detection")
+- [Detecting Source Code Plagiarism on Introductory Programming Course Assignments Using a Bytecode Approach - Oscar Karnalim](https://ieeexplore.ieee.org/abstract/document/7910274 "Detecting Source Code Plagiarism on Introductory Programming Course Assignments Using a Bytecode Approach - Oscar Karnalim")
+- [Detecting Disguised Plagiarism - Hatem A. Mahmoud](https://arxiv.org/abs/1711.02149 "Detecting Disguised Plagiarism - Hatem A. Mahmoud")
+- [Instructor-centric source code plagiarism detection and plagiarism corpus](https://dl.acm.org/doi/abs/10.1145/2325296.2325328 "Instructor-centric source code plagiarism detection and plagiarism corpus")
+
+### Steps Towards Plagiarism
+The following changes were applied to sample tasks to create test cases:
+<ul type="1">
+	<li>Inserting comments or empty lines (normalization level)</li>
+	<li>Changing variable names or function names (normalization level)</li>
+	<li>Insertion of unnecessary or changed code lines (token generation)</li>
+	<li>Changing the program flow (token generation) (statments and functions must be independent from each other)</li>
+		<ul>
+			<li>Variable decleration at the beginning of the program</li>
+			<li>Combining declerations of variables</li>
+			<li>Reuse of the same variable for other functions</li>
+		</ul>
+	<li>Changing control structures</li>
+		<ul>
+			<li>for(...) to while(...)</li>
+			<li>if(...) to switch-case</li>
+		</ul>
+	<li>Modification of expressions</li>
+		<ul>
+			<li>(X < Y) to !(X >= Y) and ++x to x = x + 1</li>
+		</ul>
+	<li>Splitting and merging statements</li>
+		<ul>
+			<li>x = getSomeValue(); y = x- z; to y = (getSomeValue() - Z;</li>
+		</ul>
+	<li>Inserting unnecessary casts</li>
+</ul>
+
+More detailed information about the create as well as about the subject to the issue can be found in the issue [Develop an end-to-end testing strategy](https://github.com/jplag/JPlag/issues/193 "Develop an end-to-end testing strategy").
+
+**The changes listed above have been developed and evaluated for purely scientific purposes and are not intended to be used for plagiarism in the public or private domain.**
+
+Software is according to [§ 2 of the copyright law](https://www.gesetze-im-internet.de/urhg/__2.html "§ 2 of the copyright law") a protected work which may not be plagiarized. 
+
+### JPlag - End To End TestSuite Structure
+The construction of an end to end test is done with the help of the JPlag api. 
+
+``` java 
+[...]
+    /**
+     * This method creates the necessary results as well as models for a test run and summarizes them for a comparison.
+     * @param testClassNames Plagiarized classes names in the resource directorie which are needed for the test
+     * @param testIdentifier name of the testId to load and identify the stored results
+     * @throws IOException is thrown in case of problems with copying the plagiarism classes
+     * @throws ExitException in case the plagiarism detection with JPlag is preemptively terminated would be of the test.
+     */
+    private void runJPlagTestSuite(String[] testClassNames, int testIdentifier) throws IOException, ExitException {
+        String functionName = StackWalker.getInstance().walk(stream -> stream.skip(1).findFirst().get()).getMethodName();
+        TestCaseModel testCaseModel = jplagTestSuiteHelper.createNewTestCase(testClassNames, functionName);
+        JPlagResult jplagResult = new JPlag(testCaseModel.getJPlagOptionsFromCurrentModel()).run();
+
+        for (JPlagComparison jPlagComparison : jplagResult.getAllComparisons()) {
+            assertEquals(testCaseModel.getCurrentJsonModel().getResultModelById(testIdentifier).getResultSimilarity(), jPlagComparison.similarity(),
+                    "The JPlag results [similarity] do not match the stored values!");
+        }
+    }
+[...]
+```
+The created plagiarisms are copied with the JPlagTestSuiteHelper into temporary directories, which can then be checked with JPlag. 
+
+In order to be able to distinguish in which domain of the recognition changes have occurred, fine granular test cases are used. These are composed of the changes already mentioned above. The plagiarism is compared with the original delivery and thus it is possible to detect and test small sections of the recognition. 
+
+The values compared so far to detect a match are limited to the similarity of the JPlag scan. It is already planned to expand the comparative values in order to have more indications of a change. 
+
+The Current Discussion is already taking place and can be found at [End to end testing - "comparative values"](https://github.com/jplag/JPlag/issues/548 "End to end testing - \"comparative values\""). 
+
+The current JPlag scans will be compared with the stored ones.
+This was done by storing the data in a *.json file which is read at the beginning of each test run. 
+
+``` json  
+{
+[...]
+    "function_name": "normalizationLevelTest",
+    "test_results": [
+      {
+        "result_similarity": 100,
+        "test_identifier": 0
+      },
+      {
+        "result_similarity": 100,
+        "test_identifier": 1
+      },
+      {
+        "result_similarity": 100,
+        "test_identifier": 2
+      }
+    ]
+  }
+[...]
+```

jplag.endToEndTesting/pom.xml

@@ -0,0 +1,29 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+    <parent>
+        <groupId>de.jplag</groupId>
+        <artifactId>aggregator</artifactId>
+        <version>${revision}</version>
+    </parent>
+    <artifactId>endToEndTesting</artifactId>
+
+    <dependencies>
+        <dependency>
+            <groupId>de.jplag</groupId>
+            <artifactId>jplag</artifactId>
+        </dependency>
+        <dependency>
+            <groupId>de.jplag</groupId>
+            <artifactId>frontend-testutils</artifactId>
+            <version>${revision}</version>
+            <type>test-jar</type>
+            <scope>test</scope>
+        </dependency>
+        <dependency>
+            <groupId>com.fasterxml.jackson.core</groupId>
+            <artifactId>jackson-databind</artifactId>
+            <version>2.13.3</version>
+        </dependency>
+    </dependencies>
+</project>

jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/constants/TestDirectoryConstants.java

@@ -0,0 +1,32 @@
+package de.jplag.end_to_end_testing.constants;
+
+import java.nio.file.Path;
+
+/**
+ * All constant values that are needed in the test cases or helper classes.
+ */
+public final class TestDirectoryConstants {
+
+    private TestDirectoryConstants() {
+        // private constructor to prevent instantiation
+    }
+
+    /**
+     * Name for the folder to copy the submissions to in order to test them with JPlag
+     */
+    public static final String TEMPORARY_DIRECTORY_NAME = "testing-directory-submission";
+    /**
+     * Create the complete path to the submission files. Here the temporary system path is extended with the
+     * "TEMPORARY_DIRECTORY_NAME", which is predefined in this class.
+     */
+    public static final String TEMPORARY_SUBMISSION_DIRECTORY_NAME = Path.of("target", TEMPORARY_DIRECTORY_NAME).toString();
+
+    /**
+     * Base path to the created plagiarism and the main file located in the project resources.
+     */
+    public static final Path BASE_PATH_TO_JAVA_RESOURCES_SORTALGO = Path.of("src", "test", "resources", "java", "sortAlgo");
+    /**
+     * Base path to the saved results of the previous tests in a *.json file
+     */
+    public static final Path BASE_PATH_TO_JAVA_RESULT_JSON = Path.of("src", "test", "resources", "results", "JavaResult.json");
+}

jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/helper/JPlagTestSuiteHelper.java

@@ -0,0 +1,120 @@
+package de.jplag.end_to_end_testing.helper;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.StandardCopyOption;
+import java.util.Arrays;
+import java.util.List;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import de.jplag.end_to_end_testing.constants.TestDirectoryConstants;
+import de.jplag.end_to_end_testing.model.JsonModel;
+import de.jplag.end_to_end_testing.model.TestCaseModel;
+import de.jplag.options.LanguageOption;
+
+/**
+ * This helper class deals with creating the test cases as well as copying and deleting for the end-to-end tests. The
+ * required plagiarisms are copied from the resource folder to a temporary location, thus creating a folder structure
+ * that can be tested by JPlag. Models are instantiated here and required information for the tests is loaded.
+ */
+public class JPlagTestSuiteHelper {
+
+    private static final Logger logger = LoggerFactory.getLogger(JPlagTestSuiteHelper.class);
+
+    private String[] resourceNames;
+    private List<JsonModel> resultModel;
+    private LanguageOption languageOption;
+
+    /**
+     * Helper class for the endToEnd tests. In this class the necessary resources are loaded, prepared and copied for the
+     * tests based on the passed parameters. An instance of this class loads all necessary paths and properties for a test
+     * run with the specified language
+     * @param languageOption for loading language-specific resources
+     * @throws IOException is thrown for all problems that may occur while parsing the json file. This includes both reading
+     * and parsing problems.
+     */
+    public JPlagTestSuiteHelper(LanguageOption languageOption) throws IOException {
+        this.languageOption = languageOption;
+        this.resourceNames = new File(TestDirectoryConstants.BASE_PATH_TO_JAVA_RESOURCES_SORTALGO.toString()).list();
+
+        this.resultModel = JsonHelper.getResultModelFromPath();
+        logger.debug("temp path at [{}]", TestDirectoryConstants.TEMPORARY_SUBMISSION_DIRECTORY_NAME);
+    }
+
+    /**
+     * creates all necessary folder paths and objects for a test run. Also searches for the stored previous results of the
+     * test in order to compare them with the current results.
+     * @param classNames Array of class names with language specific extension to be prepared for a test.
+     * @return comparison results saved for the test
+     * @throws IOException Exception can be thrown in cases that involve reading, copying or locating files.
+     */
+    public TestCaseModel createNewTestCase(String[] classNames, String functionName) throws IOException {
+        createNewTestCaseDirectory(classNames);
+        JsonModel resultJsonModel = resultModel.stream().filter(jsonModel -> functionName.equals(jsonModel.getFunctionName())).findAny().orElse(null);
+        return new TestCaseModel(TestDirectoryConstants.TEMPORARY_SUBMISSION_DIRECTORY_NAME, resultJsonModel, languageOption);
+    }
+
+    /**
+     * The copied data should be deleted after instance closure
+     * @throws IOException if an I/O error occurs
+     */
+    public void clear() throws IOException {
+        logger.debug("Class instance was cleaned!");
+        deleteCopiedFiles(new File(TestDirectoryConstants.TEMPORARY_SUBMISSION_DIRECTORY_NAME));
+    }
+
+    /**
+     * Copies the passed filenames to a temporary path to use them in the tests
+     * @throws IOException Exception can be thrown in cases that involve reading, copying or locating files.
+     */
+    private void createNewTestCaseDirectory(String[] classNames) throws IOException {
+        // before copying files to the test path, check if all files are in the resource
+        // directory
+        for (String className : classNames) {
+            if (!Arrays.asList(resourceNames).contains(className)) {
+                throw new FileNotFoundException(String.format("The specified class could not be found! [%s]", className));
+            }
+        }
+        // Copy the resources data to the temporary path
+        for (int counter = 0; counter < classNames.length; counter++) {
+            Path originalPath = Path.of(TestDirectoryConstants.BASE_PATH_TO_JAVA_RESOURCES_SORTALGO.toString(), classNames[counter]);
+            Path copiePath = Path.of(TestDirectoryConstants.TEMPORARY_SUBMISSION_DIRECTORY_NAME,
+                    TestDirectoryConstants.TEMPORARY_DIRECTORY_NAME + (counter + 1), classNames[counter]);
+
+            File directory = new File(copiePath.toString());
+            if (!directory.exists()) {
+                directory.mkdirs();
+            }
+
+            Files.copy(originalPath, copiePath, StandardCopyOption.REPLACE_EXISTING);
+            logger.debug("Copy file from [{}] to [{}]", originalPath, copiePath);
+        }
+    }
+
+    /**
+     * Delete directory with including files
+     * @param file Path to a folder or file to be deleted. This happens recursively to the path
+     * @throws IOException if an I/O error occurs
+     */
+    private void deleteCopiedFiles(File folder) throws IOException {
+        File[] files = folder.listFiles();
+        if (files != null) { // some JVMs return null for empty dirs
+            for (File file : files) {
+                if (file.isDirectory()) {
+                    deleteCopiedFiles(file);
+                } else {
+                    Files.delete(file.toPath());
+                    logger.debug("Delete file in folder: [{}]", file);
+                }
+            }
+        }
+        Files.delete(folder.toPath());
+        logger.debug("Delete folder: [{}]", folder);
+    }
+
+}

jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/helper/JsonHelper.java

@@ -0,0 +1,33 @@
+package de.jplag.end_to_end_testing.helper;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.List;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+
+import de.jplag.end_to_end_testing.constants.TestDirectoryConstants;
+import de.jplag.end_to_end_testing.model.JsonModel;
+
+/**
+ * Helper class for serializing and creating all json dependent events.
+ */
+public final class JsonHelper {
+
+    /**
+     * private constructor to prevent instantiation
+     */
+    private JsonHelper() {
+        // For Serialization
+    }
+
+    /**
+     * Parsing the old results in the json file as a list from ResultJsonModel.
+     * @return list of saved results for the test cases
+     * @throws IOException is thrown for all problems that may occur while parsing the json file. This includes both reading
+     * and parsing problems.
+     */
+    public static List<JsonModel> getResultModelFromPath() throws IOException {
+        return Arrays.asList(new ObjectMapper().readValue(TestDirectoryConstants.BASE_PATH_TO_JAVA_RESULT_JSON.toFile(), JsonModel[].class));
+    }
+}

jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/model/JsonModel.java

@@ -0,0 +1,52 @@
+package de.jplag.end_to_end_testing.model;
+
+import java.util.Arrays;
+
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+/**
+ * The ResultJsonModel is the java object for the JavaResult.json file. The object contains all the necessary
+ * information for the comparisons in the test cases between old and new results, which have been stored in the
+ * JavaResult.json file.
+ */
+public class JsonModel {
+    @JsonProperty("function_name")
+    private String functionName;
+    @JsonProperty("test_results")
+    private ResultModel[] results;
+
+    /**
+     * Constructor for the JsonModel. The model is the serialization of the Json file in the form of a Java object.
+     * @param functionName the function name for the associated test results. Used as identifier to search results for the
+     * test cases.
+     * @param results Collection of the results that are in the current json file for certain tests
+     */
+    public JsonModel(String functionName, ResultModel[] results) {
+        this.functionName = functionName;
+        this.results = results;
+    }
+
+    /**
+     * empty constructor in case the serialization contains an empty object to prevent throwing exceptions. this constructor
+     * was necessary for serialization with the Jackson parse extension
+     */
+    public JsonModel() {
+        // For Serialization
+    }
+
+    /**
+     * @return the name of the currently used function stored in json
+     */
+    public String getFunctionName() {
+        return functionName;
+    }
+
+    /**
+     * returns the results for the function that have been stored for the given id.
+     * @param identifier for the comparative values
+     * @return associated comparison values that have been assigned to the identifier
+     */
+    public ResultModel getResultModelById(Integer identifier) {
+        return Arrays.asList(results).stream().filter(resultModel -> identifier.equals(resultModel.getTestId())).findAny().orElse(null);
+    }
+}

jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/model/ResultModel.java

@@ -0,0 +1,48 @@
+package de.jplag.end_to_end_testing.model;
+
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+/**
+ * The ResultModel contains all information and comparison values that are important for a test and that could be parsed
+ * from the json file.
+ */
+public class ResultModel {
+
+    @JsonProperty("result_similarity")
+    private float resultSimilarity;
+    @JsonProperty("test_identifier")
+    private int testIdentifier;
+
+    /**
+     * Constructor for the ResultModel. The model is the serialization of the Json file in the form of a Java object.
+     * @param resultSimilarity stored comparative value of the similarity
+     * @param testIdentifier specifies which associated test results are needed for a test case and are therefore associated
+     * with this Id.
+     */
+    public ResultModel(float resultSimilarity, int testIdentifier) {
+        this.resultSimilarity = resultSimilarity;
+        this.testIdentifier = testIdentifier;
+    }
+
+    /**
+     * empty constructor in case the serialization contains an empty object to prevent throwing exceptions. this constructor
+     * was necessary for serialization with the Jackson parse extension
+     */
+    public ResultModel() {
+        // For Serialization
+    }
+
+    /**
+     * @return Identifier assigned to the result
+     */
+    public int getTestId() {
+        return testIdentifier;
+    }
+
+    /**
+     * @return of the comparative similarity
+     */
+    public float getResultSimilarity() {
+        return resultSimilarity;
+    }
+}

jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/model/TestCaseModel.java

@@ -0,0 +1,46 @@
+package de.jplag.end_to_end_testing.model;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+
+import de.jplag.options.JPlagOptions;
+import de.jplag.options.LanguageOption;
+
+/**
+ * Median class for trading the results and the paths used in the tests. Also the JPlagOptions used for a test run are
+ * created here.
+ */
+public class TestCaseModel {
+
+    private JsonModel jsonModel;
+    private LanguageOption languageOption;
+    private String submissionFolderPath;
+
+    /**
+     * Constructor for creating the instance
+     * @param submissionFolderPath path to the plagiarism folder structure to be tested using Jplag
+     * @param jsonModel the parsed result model needed for the test to be able to match it later in the tests.
+     * @param languageOption the language selection in which the tests take place
+     */
+    public TestCaseModel(String submissionFolderPath, JsonModel jsonModel, LanguageOption languageOption) {
+        this.submissionFolderPath = submissionFolderPath;
+        this.jsonModel = jsonModel;
+        this.languageOption = languageOption;
+    }
+
+    /**
+     * returns the, for the test, current result model that has been persisted in the saved list.
+     * @return ResultJsonModel loaded for the test
+     */
+    public JsonModel getCurrentJsonModel() {
+        return jsonModel;
+    }
+
+    /**
+     * creates the JPlag options for the test run from the values passed in the constructor.
+     * @return current JPlag options for the created object
+     */
+    public JPlagOptions getJPlagOptionsFromCurrentModel() {
+        return new JPlagOptions(new ArrayList<>(Arrays.asList(submissionFolderPath)), new ArrayList<>(), languageOption);
+    }
+}

jplag.endToEndTesting/src/test/java/de/jplag/end_to_end_testing/JavaEndToEndTest.java

@@ -0,0 +1,114 @@
+package de.jplag.end_to_end_testing;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.io.IOException;
+import java.util.stream.Stream;
+
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.TestInstance;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.Arguments;
+import org.junit.jupiter.params.provider.MethodSource;
+
+import de.jplag.JPlag;
+import de.jplag.JPlagComparison;
+import de.jplag.JPlagResult;
+import de.jplag.end_to_end_testing.constants.TestDirectoryConstants;
+import de.jplag.end_to_end_testing.helper.JPlagTestSuiteHelper;
+import de.jplag.end_to_end_testing.model.TestCaseModel;
+import de.jplag.exceptions.ExitException;
+import de.jplag.options.LanguageOption;
+
+/**
+ * Main test class for end-to-end testing in the Java language. The test cases aim to detect changes in the detection of
+ * plagiarism in the Java language and to be able to roughly categorize them. The plagiarism is compared with the
+ * original class. The results are compared with the results from previous tests and changes are detected.
+ */
+@TestInstance(TestInstance.Lifecycle.PER_CLASS)
+class JavaEndToEndTest {
+
+    private JPlagTestSuiteHelper jplagTestSuiteHelper;
+
+    @BeforeAll
+    public void setUp() throws IOException {
+        jplagTestSuiteHelper = new JPlagTestSuiteHelper(LanguageOption.JAVA);
+        assertTrue(TestDirectoryConstants.BASE_PATH_TO_JAVA_RESOURCES_SORTALGO.toFile().exists(), "Could not find base directory!");
+        assertTrue(TestDirectoryConstants.BASE_PATH_TO_JAVA_RESULT_JSON.toFile().isFile(), "Could not find java result json!");
+    }
+
+    @AfterEach
+    public void teardown() throws IOException {
+        // after close the created directories are deleted
+        jplagTestSuiteHelper.clear();
+    }
+
+    /**
+     * Test cases created for the normalization level
+     * @return the classes to be tested with the corresponding identifier
+     */
+    private static Stream<Arguments> normalizationLevelTestArguments() {
+        return Stream.of(Arguments.of((Object) new String[] {"SortAlgo.java", "SortAlgo1.java"}, 0),
+                Arguments.of((Object) new String[] {"SortAlgo.java", "SortAlgo2.java"}, 1),
+                Arguments.of((Object) new String[] {"SortAlgo1.java", "SortAlgo2.java"}, 2));
+    }
+
+    /**
+     * Test cases for the token generation level
+     * @return the classes to be tested with the corresponding identifier
+     */
+    private static Stream<Arguments> tokenGenerationLevelTestArguments() {
+        return Stream.of(Arguments.of((Object) new String[] {"SortAlgo.java", "SortAlgo3.java"}, 0),
+                Arguments.of((Object) new String[] {"SortAlgo.java", "SortAlgo4.java"}, 1),
+                Arguments.of((Object) new String[] {"SortAlgo.java", "SortAlgo4d1.java"}, 2));
+    }
+
+    /**
+     * Inserting comments or empty lines (normalization level) -> id = 0 Changing variable names or function names
+     * (normalization level) -> id = 1 Inserting comments or empty lines (normalization level) -> id = 2
+     * @param testClassNames Plagiarized classes names in the resource directorie which are needed for the test
+     * @param testIdentifier name of the testId to load and identify the stored results
+     * @throws IOException is thrown in case of problems with copying the plagiarism classes
+     * @throws ExitException in case the plagiarism detection with JPlag is preemptively terminated would be of the test.
+     */
+    @ParameterizedTest
+    @MethodSource("normalizationLevelTestArguments")
+    void normalizationLevelTest(String[] testClassNames, int testIdentifier) throws IOException, ExitException {
+        runJPlagTestSuite(testClassNames, testIdentifier);
+    }
+
+    /**
+     * Insertion of unnecessary or changed code lines (token generation) -> id = 0 Changing the program flow (token
+     * generation) (statements and functions must be independent from each other) -> id = 1 Variable declaration at the
+     * beginning of the program (Detecting Source Code Plagiarism [...]) -> id = 2
+     * @param testClassNames Plagiarized classes names in the resource directorie which are needed for the test
+     * @param testIdentifier name of the testId to load and identify the stored results
+     * @throws IOException is thrown in case of problems with copying the plagiarism classes
+     * @throws ExitException in case the plagiarism detection with JPlag is preemptively terminated would be of the test.
+     */
+    @ParameterizedTest
+    @MethodSource("tokenGenerationLevelTestArguments")
+    void tokenGenerationLevelTest(String[] testClassNames, int testIdentifier) throws IOException, ExitException {
+        runJPlagTestSuite(testClassNames, testIdentifier);
+    }
+
+    /**
+     * This method creates the necessary results as well as models for a test run and summarizes them for a comparison.
+     * @param testClassNames Plagiarized classes names in the resource directorie which are needed for the test
+     * @param testIdentifier name of the testId to load and identify the stored results
+     * @throws IOException is thrown in case of problems with copying the plagiarism classes
+     * @throws ExitException in case the plagiarism detection with JPlag is preemptively terminated would be of the test.
+     */
+    private void runJPlagTestSuite(String[] testClassNames, int testIdentifier) throws IOException, ExitException {
+        String functionName = StackWalker.getInstance().walk(stream -> stream.skip(1).findFirst().get()).getMethodName();
+        TestCaseModel testCaseModel = jplagTestSuiteHelper.createNewTestCase(testClassNames, functionName);
+        JPlagResult jplagResult = new JPlag(testCaseModel.getJPlagOptionsFromCurrentModel()).run();
+
+        for (JPlagComparison jPlagComparison : jplagResult.getAllComparisons()) {
+            assertEquals(testCaseModel.getCurrentJsonModel().getResultModelById(testIdentifier).getResultSimilarity(), jPlagComparison.similarity(),
+                    "The JPlag results [similarity] do not match the stored values!");
+        }
+    }
+}

jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo.java

@@ -0,0 +1,36 @@
+package SortAlgos;
+public class SortAlgo {
+
+	public void BubbleSortRecursion(Integer arr[], int n) {
+		if (n == 1) 
+		{
+			return;
+		}
+
+		for (int i = 0; i < n - 1; i++)
+		{
+			if (arr[i] > arr[i + 1]) 
+			{ 
+				swap(arr, i , i+1);
+			}
+		}
+		BubbleSortRecursion(arr, n - 1);
+	}
+
+	public void BubbleSortWithoutRecursion(Integer arr[]) {
+		for(int i = arr.length; i > 1 ; i--) {
+			for(int innerCounter = 0; innerCounter < arr.length-1; innerCounter++)
+			{
+				if (arr[innerCounter] > arr[innerCounter + 1]) {
+					swap(arr, innerCounter, (innerCounter + 1));
+				}
+			}
+		}
+	}
+
+	private final <T> void swap(T[] arr, int i, int j) {
+		T t = arr[i];
+		arr[i] = arr[j];
+		arr[j] = t;
+	}
+}

jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo1.java

@@ -0,0 +1,63 @@
+package SortAlgos;
+
+/*
+ * Unnecessary comment
+ * Unnecessary comment
+ * Unnecessary comment
+ * */
+public class SortAlgo1 {
+	//Unnecessary comment
+	public void BubbleSortRecursion(Integer arr[], int n) {
+		
+		if (n == 1) 
+		{
+			
+			return;
+			
+		}
+
+		for (int i = 0; i < n - 1; i++)
+		{
+			
+			if (arr[i] > arr[i + 1]) 
+				
+			{ 
+				
+				swap(arr, i , i+1);
+				
+			}
+		}
+		
+		BubbleSortRecursion(arr, n - 1);
+		
+	}
+	
+/*
+	
+	Unnecessary comment
+	*/
+	public void BubbleSortWithoutRecursion(Integer arr[]) {
+		//Unnecessary comment
+		for(int i = arr.length; i > 1 ; i--) {
+			
+			for(int innerCounter = 0; innerCounter < arr.length-1; innerCounter++)
+				
+			{
+				
+				if (arr[innerCounter] > arr[innerCounter + 1]) {
+					swap(arr, innerCounter, (innerCounter + 1));
+					
+				}
+			}
+		}
+	}
+
+	////Unnecessary comment
+	private final <T> void swap(T[] arr, int i, int j) {
+		T t = arr[i];
+		
+		arr[i] = arr[j];
+		
+		arr[j] = t;
+	}
+}

jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo1_2.java

@@ -0,0 +1,79 @@
+package SortAlgos;
+
+public class SortAlgo1_2 {
+	//Unnecessary comment
+	public void ChangeFunctionName(Integer changeVariableArray[], int counter) {
+		
+		/*  
+		 * Comment
+		 * */
+		if (counter == 1) 
+		{
+			return;
+		}
+
+		for (int i = 0; i < counter - 1; i++)
+		{
+			
+			
+			if (changeVariableArray[i] > changeVariableArray[i + 1]) 
+			{ 
+				
+				
+				paws(changeVariableArray, i , i+1);
+				
+				
+			}
+		}
+		ChangeFunctionName(changeVariableArray, counter - 1);
+	}
+		
+	/*
+		
+		Unnecessary comment
+		*/
+	public void ChangeFunctionNameWithoutRecursion(Integer arr[]) {
+		//Unnecessary comment
+		for(int otherVariable = arr.length; otherVariable > 1 ; otherVariable--) {
+			
+			
+			for(int counter = 0; counter < arr.length-1; counter++)
+				
+			{
+				if (arr[counter] > arr[counter + 1]) {
+					paws(arr, counter, (counter + 1));
+					
+				}
+				
+			}
+		}
+	}
+		public void BubbleSortWithoutRecursion(Integer arr[]) {
+			//Unnecessary comment
+			for(int i = arr.length; i > 1 ; i--) {
+				
+				for(int innerCounter = 0; innerCounter < arr.length-1; innerCounter++)
+					
+				{
+					
+					if (arr[innerCounter] > arr[innerCounter + 1]) {
+						paws(arr, innerCounter, (innerCounter + 1));
+						
+					}
+					
+				}
+			}
+		}
+
+		////Unnecessary comment
+		private final <T> void paws(T[] arr, int i, int j) {
+			T t = arr[i];
+			//Comment
+			arr[i] = arr[j];
+			//Comment
+			arr[j] = t;
+		}
+}
+
+
+//Comment

jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo1_3.java

@@ -0,0 +1,81 @@
+package SortAlgos;
+
+public class SortAlgo1_3 {
+	
+	private boolean _notUsedBool = false;
+	private int _thisIsNotUsed = 0;
+//Unnecessary comment
+	public void BubbleSortRecursion(Integer arr[], int n) {
+		if (_thisIsNotUsed == 0 && !_notUsedBool)
+			if (n == 1) {
+				return;
+			}
+		/*
+		 * Unnecessary comment
+		 * */
+
+		var tempVar = 0;
+		//Comment
+		var tempVar1 = tempVar + 1;
+		//Comment
+		var tempVar2 = tempVar1 + 1;
+		
+		for (int i = 0; i < n - 1; i++) {
+			
+			/*
+			 * Comment
+			 * Comment
+			 * */
+			if (arr[i] > arr[i + 1]) {
+				
+				
+				swap(arr, i, i + 1);
+				
+			}
+		}
+		
+		
+		//Unnecessary comment
+		BubbleSortRecursion(arr, n - 1);
+	}
+/*
+ * Unnecessary comment
+ * Unnecessary comment
+ * */
+	public void BubbleSortWithoutRecursion(Integer arr[]) {
+		//Unnecessary comment
+		var tempVar = 0;
+		var tempVar1 = tempVar + 1;
+		tempVar1 = tempVar1 + 1;
+		tempVar = tempVar1 *1;
+		
+		for (int i = arr.length; i > 1; i--) {
+			//Comment
+			for (int innerCounter = 0; innerCounter < arr.length - 1; innerCounter++) {
+				if (arr[innerCounter] > arr[innerCounter + 1]) {
+					
+					swap(arr, innerCounter, (innerCounter + 1));
+					
+					
+					
+				}
+			}
+		}
+	}
+
+	private final <T> void swap(T[] arr, int i, int j) {
+		var tempVar1 = 0;
+		var tempVar2 = tempVar1 + 1;
+		tempVar2 = tempVar2 + 1;
+		
+		//Unnecessary comment
+		
+		
+		if (true) {
+			T t = arr[i];
+			arr[i] = arr[j];
+			arr[j] = t;
+			tempVar2 = 0;
+		}
+	}
+}

jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo1_4.java

@@ -0,0 +1,71 @@
+package SortAlgos;
+/*
+ * Unnecessary comment
+ * Unnecessary comment
+ * Unnecessary comment
+ * */
+public class SortAlgo1_4 {
+	
+	/*
+	
+	Unnecessary comment
+	*/
+	public void BubbleSortWithoutRecursion(Integer arr[]) {
+		//Unnecessary comment
+		for(int i = arr.length; i > 1 ; i--) {
+			
+			for(int innerCounter = 0; innerCounter < arr.length-1; innerCounter++)
+				
+			{
+				
+				if (arr[innerCounter] > arr[innerCounter + 1]) {
+					swap(arr, innerCounter, (innerCounter + 1));
+					
+				}
+			}
+		}
+	}
+	
+
+	////Unnecessary comment
+	private final <T> void swap(T[] arr, int i, int j) {
+		T t = arr[i];
+		
+		arr[i] = arr[j];
+		
+		arr[j] = t;
+	}
+	
+	//Unnecessary comment
+		public void BubbleSortRecursion(Integer arr[], int n) {
+			
+
+			for (int i = 0; i < n - 1; i++)
+			{
+				
+				if (arr[i] > arr[i + 1]) 
+					
+				{ 
+					
+					swap(arr, i , i+1);
+					
+				}
+			}
+			
+			
+			if (n == 2) {
+				
+				return;
+				
+			} else {
+				
+				
+				BubbleSortRecursion(arr, n - 1);
+				
+			}
+			
+		}
+		
+	
+
+}

jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo1_5.java

@@ -0,0 +1,91 @@
+package SortAlgos;
+/*
+ * Unnecessary comment
+ * Unnecessary comment
+ * Unnecessary comment
+ * */
+public class SortAlgo1_5 {
+	
+	public void BubbleSortRecursion(Integer arr[], int n) {
+		
+		switch (n) {
+		
+		case 1:
+			return;
+			
+			
+			
+		}
+	////Unnecessary comment
+	////Unnecessary comment
+	////Unnecessary comment
+		/*
+		 * 
+		 * 
+		 * 
+		 * */
+		int i = 0;
+		
+		while(i < n-1)
+			
+		{
+			var tempBool = arr[i] > arr[i + 1];
+			
+			if (tempBool) {
+				
+				swap(arr, i, i + 1);
+				
+			}
+			
+			i++;
+		}
+		
+		BubbleSortRecursion(arr, n - 1);
+		
+		
+	}
+////Unnecessary comment
+	public void BubbleSortWithoutRecursion(Integer arr[]) {
+		
+		
+		int i = arr.length;
+		
+	////Unnecessary comment
+		while(i > 1)
+		{
+		////Unnecessary comment
+			int innerCounter = 0;
+			
+			
+			while(innerCounter < arr.length -1)
+			////Unnecessary comment
+			{
+				if (arr[innerCounter] > arr[innerCounter + 1]) {
+					
+					
+					swap(arr, innerCounter, (innerCounter + 1));
+					
+				}
+				
+				
+				innerCounter++;
+				
+				
+			}
+			i--;
+		}
+	}
+////Unnecessary comment
+	private final <T> void swap(T[] arr, int i, int j) {
+		
+	////Unnecessary comment////Unnecessary comment
+		T t = arr[i];
+		
+		
+	////Unnecessary comment
+		arr[i] = arr[j];
+		
+	////Unnecessary comment
+		arr[j] = t;
+	}
+}

jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo1_6.java

@@ -0,0 +1,91 @@
+package SortAlgos;
+
+/*
+ * Unnecessary comment
+ * Unnecessary comment
+ * Unnecessary comment
+ * */
+
+
+public class SortAlgo1_6 {
+	
+	//
+	//
+	//
+	//
+	public void BubbleSortRecursion(Integer arr[], int n) {
+		/*
+		 * Unnecessary comment
+		 * Unnecessary comment
+		 * Unnecessary comment
+		 * */
+		if (n != 1) 
+		{
+			//
+			
+			for (int i = 0; !(i >= (n - 1));)
+			{
+				
+				
+				if (!(arr[i] <= arr[i + 1])) 
+				{ 
+					
+					
+					swap(arr, i , i+1);
+				}
+				
+				
+				i = i + 1;
+			}
+			
+			
+			BubbleSortRecursion(arr, n - 1);
+		}
+		else
+			
+			
+		{
+			return;
+		}
+	}
+
+	/*
+	 * Unnecessary comment
+	 * Unnecessary comment
+	 * Unnecessary comment
+	 * */
+	public void BubbleSortWithoutRecursion(Integer arr[]) {
+		
+		for(int i = arr.length; !(i < 1) ; i--) {
+			
+			////Comment
+			//Comment
+			//Comment
+			
+			for(int innerCounter = 0; innerCounter < arr.length-1; innerCounter++)
+			{
+				
+				//Comment
+				if (!(arr[innerCounter] <= arr[innerCounter + 1])) {
+					
+					
+					swap(arr, innerCounter, (innerCounter + 1));
+					
+					    
+				}
+			}
+		}
+	}
+
+	private final <T> void swap(T[] arr, int i, int j) {
+		
+		T t = arr[i];
+		
+		
+		arr[i] = arr[j];
+		
+		arr[j] = t;
+		
+		
+	}
+}

jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo2.java

@@ -0,0 +1,38 @@
+package SortAlgos;
+
+
+public class SortAlgo2 {
+
+	public void ChangeFunctionName(Integer changeVariableArray[], int counter) {
+		if (counter == 1) 
+		{
+			return;
+		}
+
+		for (int i = 0; i < counter - 1; i++)
+		{
+			if (changeVariableArray[i] > changeVariableArray[i + 1]) 
+			{ 
+				paws(changeVariableArray, i , i+1);
+			}
+		}
+		ChangeFunctionName(changeVariableArray, counter - 1);
+	}
+
+	public void ChangeFunctionNameWithoutRecursion(Integer arr[]) {
+		for(int otherVariable = arr.length; otherVariable > 1 ; otherVariable--) {
+			for(int counter = 0; counter < arr.length-1; counter++)
+			{
+				if (arr[counter] > arr[counter + 1]) {
+					paws(arr, counter, (counter + 1));
+				}
+			}
+		}
+	}
+
+	private final <T> void paws(T[] otherArr, int i, int j) {
+		T t = otherArr[i];
+		otherArr[i] = otherArr[j];
+		otherArr[j] = t;
+	}
+}

jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo2_5.java

@@ -0,0 +1,44 @@
+package SortAlgos;
+
+public class SortAlgo2_5 {
+	public void ChangeFunctionName(Integer changeVariableArray[], int counter) {
+		switch (counter) {
+		case 1:
+			return;
+		}
+
+		int i = 0;
+		while(i < counter-1)
+		{
+			var tempBool = changeVariableArray[i] > changeVariableArray[i + 1];
+			if (tempBool) {
+				paws(changeVariableArray, i, i + 1);
+			}
+			i++;
+		}
+		
+		ChangeFunctionName(changeVariableArray, counter - 1);
+	}
+
+	public void ChangeFunctionNameWithoutRecursion(Integer arr[]) {
+		int otherVariable = arr.length;
+		while(otherVariable > 1)
+		{
+			int innerCounter = 0;
+			while(innerCounter < arr.length -1)
+			{
+				if (arr[innerCounter] > arr[innerCounter + 1]) {
+					paws(arr, innerCounter, (innerCounter + 1));
+				}
+				innerCounter++;
+			}
+			otherVariable--;
+		}
+	}
+
+	private final <T> void paws(T[] otherArr, int i, int j) {
+		T t = otherArr[i];
+		otherArr[i] = otherArr[j];
+		otherArr[j] = t;
+	}
+}

jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo3.java

@@ -0,0 +1,47 @@
+package SortAlgos;
+
+public class SortAlgo3 {
+
+	private int _thisIsNotUsed = 0;
+
+	public void BubbleSortRecursion(Integer arr[], int n) {
+		if (_thisIsNotUsed == 0)
+			if (n == 1) {
+				return;
+			}
+
+		var tempVar = 0;
+		tempVar ++;
+		tempVar = tempVar -1;
+		
+		for (int i = 0; i < n - 1; i++) {
+			if (arr[i] > arr[i + 1]) {
+				swap(arr, i, i + 1);
+			}
+		}
+		BubbleSortRecursion(arr, n - 1);
+	}
+
+	public void BubbleSortWithoutRecursion(Integer arr[]) {
+		var tempVar = 0;
+		for (int i = arr.length; i > 1; i--) {
+			for (int innerCounter = 0; innerCounter < arr.length - 1; innerCounter++) {
+				if (arr[innerCounter] > arr[innerCounter + 1]) {
+					swap(arr, innerCounter, (innerCounter + 1));
+				}
+			}
+		}
+	}
+
+	private final <T> void swap(T[] arr, int i, int j) {
+		var tempVar1 = 0;
+		if (true) {
+			T t = arr[i];
+			arr[i] = arr[j];
+			arr[j] = t;
+			var tempVar2 = 0;
+			tempVar2++;
+			tempVar2 = tempVar2 + 1;
+		}
+	}
+}

jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo3_5.java

@@ -0,0 +1,59 @@
+package SortAlgos;
+
+public class SortAlgo3_5 {
+	private int _thisIsNotUsed = 0;
+
+	public void BubbleSortRecursion(Integer arr[], int n) {
+		if (_thisIsNotUsed == 0)
+			switch (n) {
+			case 1:
+				return;
+			}
+
+		var tempVar = 0;
+		tempVar++;
+		tempVar = tempVar - 1;
+
+		int i = 0;
+		while(i < n-1)
+		{
+			var tempBool = arr[i] > arr[i + 1];
+			if (tempBool) {
+				swap(arr, i, i + 1);
+			}
+			i++;
+		}
+		
+		BubbleSortRecursion(arr, n - 1);
+	}
+
+	public void BubbleSortWithoutRecursion(Integer arr[]) {
+		int i = arr.length;
+		var tempVar = 0;
+		
+		while(i > 1)
+		{
+			int innerCounter = 0;
+			while(innerCounter < arr.length -1)
+			{
+				if (arr[innerCounter] > arr[innerCounter + 1]) {
+					swap(arr, innerCounter, (innerCounter + 1));
+				}
+				innerCounter++;
+			}
+			i--;
+		}
+	}
+
+	private final <T> void swap(T[] arr, int i, int j) {
+		var tempVar1 = 0;
+		if (true) {
+			T t = arr[i];
+			var tempVar2 = 0;
+			arr[i] = arr[j];
+			arr[j] = t;
+			tempVar2++;
+			tempVar2 = tempVar2 + 1;
+		}
+	}
+}

jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo3_6.java

@@ -0,0 +1,54 @@
+package SortAlgos;
+
+public class SortAlgo3_6 {
+	private int _thisIsNotUsed = 0;
+
+	public void BubbleSortRecursion(Integer arr[], int n) {
+		if (_thisIsNotUsed == 0) {
+			if (n != 1) {
+
+				var tempVar = 0;
+				tempVar++;
+				tempVar = tempVar - 1;
+
+				for (int i = 0; !(i >= (n - 1));)
+				{
+					if (!(arr[i] <= arr[i + 1])) 
+					{ 
+						swap(arr, i , i+1);
+					}
+					i = i + 1;
+				}
+				
+				BubbleSortRecursion(arr, n - 1);
+			} else {
+				return;
+			}
+
+		}
+	}
+
+	public void BubbleSortWithoutRecursion(Integer arr[]) {
+		var tempVar = 0;
+		for(int i = arr.length; !(i < 1) ; i--) {
+			for(int innerCounter = 0; innerCounter < arr.length-1; innerCounter++)
+			{
+				if (!(arr[innerCounter] <= arr[innerCounter + 1])) {
+					swap(arr, innerCounter, (innerCounter + 1));
+				}
+			}
+		}
+	}
+
+	private final <T> void swap(T[] arr, int i, int j) {
+		var tempVar1 = 0;
+		if (true) {
+			T t = arr[i];
+			arr[i] = arr[j];
+			arr[j] = t;
+			var tempVar2 = 0;
+			tempVar2++;
+			tempVar2 = tempVar2 + 1;
+		}
+	}
+}

jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo4.java

@@ -0,0 +1,33 @@
+package SortAlgos;
+
+public class SortAlgo4 {
+	public void BubbleSortWithoutRecursion(Integer arr[]) {
+		for (int i = arr.length; i > 1; i--) {
+			for (int innerCounter = 0; innerCounter < arr.length - 1; innerCounter++) {
+				if (arr[innerCounter] > arr[innerCounter + 1]) {
+					swap(arr, innerCounter, (innerCounter + 1));
+				}
+			}
+		}
+	}
+
+	public void BubbleSortRecursion(Integer arr[], int n) {
+		for (int i = 0; i < n - 1; i++) {
+			if (arr[i] > arr[i + 1]) {
+				swap(arr, i, i + 1);
+			}
+		}
+
+		if (n == 2) {
+			return;
+		} else {
+			BubbleSortRecursion(arr, n - 1);
+		}
+	}
+
+	private final <T> void swap(T[] arr, int i, int j) {
+		T t = arr[i];
+		arr[i] = arr[j];
+		arr[j] = t;
+	}
+}

jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo4d1.java

@@ -0,0 +1,40 @@
+public class SortAlgo2 {
+	private int firstCounter;
+	private int arrayLenght;
+	private int swapVarI;
+	private int swapVarJ;
+
+	public void BubbleSortRecursion(Integer arr[], int n) {
+		firstCounter = n;
+		if (firstCounter == 1) {
+			return;
+		}
+
+		for (int i = 0; i < firstCounter - 1; i++) {
+			if (arr[i] > arr[i + 1]) {
+				swap(arr, i, i + 1);
+			}
+		}
+		BubbleSortRecursion(arr, firstCounter - 1);
+	}
+
+	public void BubbleSortWithoutRecursion(Integer arr[]) {
+		arrayLenght = arr.length;
+		for (int i = arrayLenght; i > 1; i--) {
+			for (int innerCounter = 0; innerCounter < arrayLenght - 1; innerCounter++) {
+				if (arr[innerCounter] > arr[innerCounter + 1]) {
+					swap(arr, innerCounter, (innerCounter + 1));
+				}
+			}
+		}
+	}
+
+	private final <T> void swap(T[] arr, int i, int j) {
+		swapVarI = i;
+		swapVarJ = j;
+
+		T t = arr[swapVarI];
+		arr[swapVarI] = arr[swapVarJ];
+		arr[swapVarJ] = t;
+	}
+}

jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo4d2.java

@@ -0,0 +1,39 @@
+package SortAlgos;
+
+public class SortAlgo4d2 {
+	private int firstCounter,swapVarJ,arrayLenght ,swapVarI;
+
+	public void BubbleSortRecursion(Integer arr[], int n) {
+		firstCounter = n;
+		if (firstCounter == 1) {
+			return;
+		}
+
+		for (int i = 0; i < firstCounter - 1; i++) {
+			if (arr[i] > arr[i + 1]) {
+				swap(arr, i, i + 1);
+			}
+		}
+		BubbleSortRecursion(arr, firstCounter - 1);
+	}
+
+	public void BubbleSortWithoutRecursion(Integer arr[]) {
+		arrayLenght = arr.length;
+		for (int i = arrayLenght; i > 1; i--) {
+			for (int innerCounter = 0; innerCounter < arrayLenght - 1; innerCounter++) {
+				if (arr[innerCounter] > arr[innerCounter + 1]) {
+					swap(arr, innerCounter, (innerCounter + 1));
+				}
+			}
+		}
+	}
+
+	private final <T> void swap(T[] arr, int i, int j) {
+		swapVarI = i;
+		swapVarJ = j;
+
+		T t = arr[swapVarI];
+		arr[swapVarI] = arr[swapVarJ];
+		arr[swapVarJ] = t;
+	}
+}

jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo4d3.java

@@ -0,0 +1,39 @@
+package SortAlgos;
+
+public class SortAlgo4d3 {
+	private int firstCounterAndArrayLenghtAndswapVarJ ,swapVarI;
+
+	public void BubbleSortRecursion(Integer arr[], int n) {
+		firstCounterAndArrayLenghtAndswapVarJ = n;
+		if (firstCounterAndArrayLenghtAndswapVarJ == 1) {
+			return;
+		}
+
+		for (int i = 0; i < firstCounterAndArrayLenghtAndswapVarJ - 1; i++) {
+			if (arr[i] > arr[i + 1]) {
+				swap(arr, i, i + 1);
+			}
+		}
+		BubbleSortRecursion(arr, firstCounterAndArrayLenghtAndswapVarJ - 1);
+	}
+
+	public void BubbleSortWithoutRecursion(Integer arr[]) {
+		firstCounterAndArrayLenghtAndswapVarJ = arr.length;
+		for (int i = firstCounterAndArrayLenghtAndswapVarJ; i > 1; i--) {
+			for (int innerCounter = 0; innerCounter < firstCounterAndArrayLenghtAndswapVarJ - 1; innerCounter++) {
+				if (arr[innerCounter] > arr[innerCounter + 1]) {
+					swap(arr, innerCounter, (innerCounter + 1));
+				}
+			}
+		}
+	}
+
+	private final <T> void swap(T[] arr, int i, int j) {
+		swapVarI = i;
+		firstCounterAndArrayLenghtAndswapVarJ = j;
+
+		T t = arr[swapVarI];
+		arr[swapVarI] = arr[firstCounterAndArrayLenghtAndswapVarJ];
+		arr[firstCounterAndArrayLenghtAndswapVarJ] = t;
+	}
+}

jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo5.java

@@ -0,0 +1,44 @@
+package SortAlgos;
+
+public class SortAlgo5 {
+	public void BubbleSortRecursion(Integer arr[], int n) {
+		switch (n) {
+		case 1:
+			return;
+		}
+
+		int i = 0;
+		while(i < n-1)
+		{
+			var tempBool = arr[i] > arr[i + 1];
+			if (tempBool) {
+				swap(arr, i, i + 1);
+			}
+			i++;
+		}
+		
+		BubbleSortRecursion(arr, n - 1);
+	}
+
+	public void BubbleSortWithoutRecursion(Integer arr[]) {
+		int i = arr.length;
+		while(i > 1)
+		{
+			int innerCounter = 0;
+			while(innerCounter < arr.length -1)
+			{
+				if (arr[innerCounter] > arr[innerCounter + 1]) {
+					swap(arr, innerCounter, (innerCounter + 1));
+				}
+				innerCounter++;
+			}
+			i--;
+		}
+	}
+
+	private final <T> void swap(T[] arr, int i, int j) {
+		T t = arr[i];
+		arr[i] = arr[j];
+		arr[j] = t;
+	}
+}

jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo6.java

@@ -0,0 +1,45 @@
+package SortAlgos;
+
+public class SortAlgo6 {
+	public void BubbleSortRecursion(Integer arr[], int n) {
+		if (n != 1) 
+		{
+			for (int i = 0; !(i >= (n - 1));)
+			{
+				if (!(arr[i] <= arr[i + 1])) 
+				{ 
+					swap(arr, i , i+1);
+				}
+				i = i + 1;
+			}
+			BubbleSortRecursion(arr, n - 1);
+		}
+		else
+		{
+			return;
+		}
+	}
+
+	public void BubbleSortWithoutRecursion(Integer arr[]) {
+		for(int i = arr.length; !(i < 1) ; i--) {
+			for(int innerCounter = 0; innerCounter < arr.length-1; innerCounter++)
+			{
+				if (!(arr[innerCounter] <= arr[innerCounter + 1])) {
+					swap(arr, innerCounter, (innerCounter + 1));
+				}
+			}
+		}
+	}
+
+	private final <T> void swap(T[] arr, int i, int j) {
+		var tempVar1 = 0;
+		if (true) {
+			T t = arr[i];
+			arr[i] = arr[j];
+			arr[j] = t;
+			var tempVar2 = 0;
+			tempVar2++;
+			tempVar2 = tempVar2 + 1;
+		}
+	}
+}

jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo7.java

@@ -0,0 +1,33 @@
+package SortAlgos;
+/*
+ * No able changes for (Splitting and merging statements)
+ * */
+public class SortAlgo7 {
+	public void BubbleSortRecursion(Integer arr[], int n) {
+		if (n == 1)
+			return;
+
+		for (int i = 0; i < n - 1; i++) {
+			if (arr[i] > arr[i + 1]) {
+				swap(arr, i, i + 1);
+			}
+		}
+		BubbleSortRecursion(arr, n - 1);
+	}
+
+	public void BubbleSortWithoutRecursion(Integer arr[]) {
+		for (int i = arr.length; i > 1; i--) {
+			for (int innerCounter = 0; innerCounter < arr.length - 1; innerCounter++) {
+				if (arr[innerCounter] > arr[innerCounter + 1]) {
+					swap(arr, innerCounter, (innerCounter + 1));
+				}
+			}
+		}
+	}
+
+	private final <T> void swap(T[] arr, int i, int j) {
+		T t = arr[i];
+		arr[i] = arr[j];
+		arr[j] = t;
+	}
+}

jplag.endToEndTesting/src/test/resources/results/JavaResult.json

@@ -0,0 +1,36 @@
+[
+  {
+    "function_name": "normalizationLevelTest",
+    "test_results": [
+      {
+        "result_similarity": 100,
+        "test_identifier": 0
+      },
+      {
+        "result_similarity": 100,
+        "test_identifier": 1
+      },
+      {
+        "result_similarity": 100,
+        "test_identifier": 2
+      }
+    ]
+  },
+  {
+    "function_name": "tokenGenerationLevelTest",
+    "test_results": [
+      {
+        "result_similarity": 64.0,
+        "test_identifier": 0
+      },
+      {
+        "result_similarity": 73.04348,
+        "test_identifier": 1
+      },
+      {
+        "result_similarity": 73.94958,
+        "test_identifier": 2
+      }
+    ]
+  }
+]

jplag.frontend.scala/README.md

@@ -0,0 +1,92 @@
+# JPlag Scala language frontend
+
+The JPlag Scala frontend allows the use of JPlag with submissions in Scala. <br>
+It is based on the [Scalameta library](https://scalameta.org/) parser, and is adapted from
+the [CodeGra-de Scala frontend](https://github.com/CodeGra-de/jplag/tree/master/jplag.frontend.scala) for JPlag, both
+licensed under BSD-3.
+
+### Scala specification compatibility
+
+The dependencies only allow compatibility up to Scala 2.13.8 (January 2022), so the new syntactical features of Scala 3 
+are not supported yet.
+
+As of now, Scalameta is not available for Scala 3 yet (see the [GitHub issue](https://github.com/scalameta/scalameta/issues/2485)), 
+so the upgrade needs to wait. It would seem that once this frontend is equipped with Scalameta for Scala 3, it will be able to handle both Scala 2 and 3 equally as [the syntax is backwards compatible](https://scala-lang.org/2019/12/18/road-to-scala-3.html#:~:text=Scala%203%20is%20backwards%20compatible%20with%20Scala%202) for the most part.
+
+### Token Extraction
+
+#### General
+
+The choice of tokens is intended to be similar to the Java or C# frontends. Specifically, among others, it includes a
+range of nesting structures (class and method declarations, control flow expressions) as well as variable declaration,
+object creation, assignment, and control flow altering keywords. <br>
+Blocks are distinguished by their context, i.e. there are separate `TokenConstants` for `if` blocks, `for` blocks, class
+bodies, method bodies, array constructors, and the like.
+
+More syntactic elements of Scala may turn out to be helpful to include in the future, especially those that are newly
+introduced.
+
+#### Problem in Scala (1): Method calls
+
+The syntax of Scala allows to omit the parentheses when calling methods without arguments. These method calls are
+indistinguishable from member references. This makes the system vulnerable to attacks where an empty set of brackets is
+simply added after the member name. To address this, method calls with no arguments are not marked with an `APPLY`
+token, even though they are recognizable as method calls.
+
+```scala
+myObject.member // may be member reference or method call
+// gets MEMBER token
+
+myObject.member2() // must be method call
+// gets MEMBER token
+
+myObject.member3(arg1, arg2) // must be method call
+// gets APPLY MEMBER ARG ARG tokens
+```
+
+#### Problem in Scala (2): Operators
+
+Operators are implemented as regular method calls. Additionally, custom operators on objects/classes can be defined,
+possibly overloading existing ones like `+`, `&=` etc.
+
+In other frontends, operations are not assigned tokens but "regular" method calls are. This calls for the task to try to
+distinguish operations from what we understand as "regular" method calls. This is not entirely possible with only
+parsing information, so we decided to go about this problem as follows:
+
+- Calls to methods with an identifier that is used as an operator are NOT treated as a method call. This is accomplished
+  by comparing to a hard-coded list of standard operators on numbers, booleans, lists, and types (although type
+  operators cannot be used in the same contexts as the others). This applies in infix and dot notation.
+- Calls to methods with any other identifier, be it alphanumerical, symbolic or any combination, are treated as method
+  calls and are assigned `APPLY` and `ARG` tokens if applicable, see (1).
+
+#### Problem in Scala (3): `return` is optional and discouraged
+
+In Scala, the use of the `return` keyword is regarded as a bad smell because it may disrupt the control flow in ways
+unintended by the less experienced Scala developer.
+Instead, like any block of code, the method body is evaluated to the last expression that is evaluated.
+
+```scala
+def power(base: Int, exponent: Int): Int = {
+    if (exponent == 0) 1                                // mark this return value?
+    else if (exponent == 1) base                        // and this one?
+    else if (exponent % 2 == 0) 
+        ((i: Int) => i*i)(power(base, exponent / 2))    // and this one?
+    else base * power(base, exponent - 1)               // and this one?
+}
+```
+That raises the question whether to try and mark these more implicit return values, so that the output of this frontend
+would be consistent with others.
+
+To determine all possible return values, semantic information about control structures is necessary which may be tedious
+to extract from the AST, but possible (e.g. by means of a stack mechanic).
+On the other hand, "the last expression of a block evaluated" does not hold the same _syntactical_ weight to it as a return
+statement.
+
+For the moment, implicit block values are neglected.
+
+### Usage
+
+To use the Scala frontend, add the `-l scala` flag in the CLI, or use a `JPlagOption` object set
+to `LanguageOption.SCALA` in the Java API as described in the usage information in
+the [readme of the main project](https://github.com/jplag/JPlag#usage)
+and [in the wiki](https://github.com/jplag/JPlag/wiki/1.-How-to-Use-JPlag).

jplag.frontend.scala/pom.xml

@@ -0,0 +1,73 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
+    <modelVersion>4.0.0</modelVersion>
+
+    <parent>
+        <groupId>de.jplag</groupId>
+        <artifactId>aggregator</artifactId>
+        <version>${revision}</version>
+    </parent>
+    <artifactId>scala</artifactId>
+
+    <properties>
+        <scala.version>2.13.8</scala.version>
+        <scala.compat.version>2.13</scala.compat.version>
+    </properties>
+
+    <dependencies>
+        <dependency>
+            <groupId>org.scala-lang</groupId>
+            <artifactId>scala-library</artifactId>
+            <version>${scala.version}</version>
+        </dependency>
+        <dependency>
+            <groupId>de.jplag</groupId>
+            <artifactId>frontend-utils</artifactId>
+            <version>${revision}</version>
+        </dependency>
+
+        <!-- https://mvnrepository.com/artifact/org.scalameta/scalameta -->
+        <dependency>
+            <groupId>org.scalameta</groupId>
+            <artifactId>scalameta_${scala.compat.version}</artifactId>
+            <version>4.5.9</version>
+        </dependency>
+
+        <!-- Test -->
+        <dependency>
+            <groupId>de.jplag</groupId>
+            <artifactId>frontend-testutils</artifactId>
+            <version>${revision}</version>
+            <type>test-jar</type>
+            <scope>test</scope>
+        </dependency>
+    </dependencies>
+
+    <build>
+        <plugins>
+            <plugin>
+                <!-- see http://davidb.github.com/scala-maven-plugin -->
+                <groupId>net.alchim31.maven</groupId>
+                <artifactId>scala-maven-plugin</artifactId>
+                <version>3.3.2</version>
+                <executions>
+                    <execution>
+                        <goals>
+                            <goal>compile</goal>
+                            <goal>testCompile</goal>
+                        </goals>
+                        <configuration>
+                            <args>
+                                <arg>-dependencyfile</arg>
+                                <arg>${project.build.directory}/.scala_dependencies</arg>
+                            </args>
+                        </configuration>
+                    </execution>
+                </executions>
+            </plugin>
+        </plugins>
+        <sourceDirectory>src/main/scala</sourceDirectory>
+        <testSourceDirectory>src/test/java</testSourceDirectory>
+    </build>
+
+</project>

jplag.frontend.scala/src/main/scala/de/jplag/scala/Language.scala

@@ -0,0 +1,23 @@
+package de.jplag.scala
+
+import de.jplag.TokenList
+
+import java.io.File
+
+class Language extends de.jplag.Language {
+  private val parser = new Parser
+  private final val fileExtensions = Array(".scala", ".sc")
+
+  override def suffixes: Array[String] = fileExtensions
+
+  override def getName = "Scala parser"
+
+  override def getShortName = "scala"
+
+  override def minimumTokenMatch = 8
+
+  override def parse(dir: File, files: Array[String]): TokenList = this.parser.parse(dir, files)
+
+  override def hasErrors: Boolean = this.parser.hasErrors
+
+}

jplag.frontend.scala/src/main/scala/de/jplag/scala/Parser.scala

@@ -0,0 +1,394 @@
+package de.jplag.scala
+
+import de.jplag.scala.ScalaTokenConstants._
+import de.jplag.{AbstractParser, TokenList}
+
+import java.io.File
+import scala.meta._
+
+
+class Parser extends AbstractParser {
+    private var currentFile: String = _
+
+    private var tokens: TokenList = _
+
+    private val traverser: Traverser = new Traverser {
+
+        private val Operators: Array[String] = Array(
+            "+", "-", "*", "/", "%", "**",
+            "==", "!=", ">", "<", ">=", "<=",
+            "&&", "||", "!",
+            "=", "+=", "-=", "*=", "/=", "%=", "**=", "<<=", ">>=", ">>>=", "&=", "^=", "|=",
+            "&", "|", "^", "<<", ">>", "~", ">>>",
+            "++", "::", ":::",
+            "<:", ">:", "#"
+        )
+
+        /**
+         * TraverserRecord; Identifier kept short for readability
+         *
+         * @param before   opt. token type to insert at the beginning of the tree
+         * @param after    opt. token type to insert at the end of the tree
+         * @param traverse custom method of traversing the tree
+         */
+        private case class TR(before: Option[ScalaTokenConstants.Value] = None,
+                              after: Option[ScalaTokenConstants.Value] = None,
+                              traverse: Tree => Unit = _.children.foreach(traverser.apply)
+                             )
+
+        /**
+         * Adds a token at the beginning of the tree and traverses it if a tree is given.
+         *
+         * @param tree  opt. tree
+         * @param token token type to insert at the beginning of the tree
+         */
+        private def maybeAddAndApply(tree: Option[Tree], token: ScalaTokenConstants.Value): Unit = tree match {
+            case Some(expression) =>
+                add(token, expression, fromEnd = false)
+                apply(expression)
+            case None =>
+        }
+
+        private def processCases(cases: List[Case]): Unit = cases.foreach {
+            case caseTree@Case(pattern, condition, body) =>
+                add(ScalaTokenConstants.CaseStatement, caseTree, fromEnd = false)
+                applyRecursively(Seq(pattern, condition))
+
+                encloseAndApply(body, TR(Some(CaseBegin), Some(CaseEnd)))
+            case _ =>
+        }
+
+        private def isStandardOperator(operator: String): Boolean = {
+            Operators.contains(operator)
+        }
+
+        private def getMethodIdentifier(function: Term): String = {
+            function.toString().split("\\.").last
+        }
+
+        private def handleDefinitionPattern(pattern: Pat, optionalValue: Option[Term]): Unit = {
+            pattern match {
+                // variable tuple
+                case tuple@Pat.Tuple(patternArgs) =>
+                    optionalValue match {
+                        // not initialized
+                        case None => for (elem <- patternArgs) handleDefinitionPattern(elem, None)
+
+                        // initialized with tuple literal
+                        case Some(Term.Tuple(values)) => patternArgs.zip(values).foreach {
+                            case (pattern, value) => handleDefinitionPattern(pattern, Some(value))
+                        }
+
+                        // initialized with tuple reference
+                        case Some(value) =>
+                            for (elem <- patternArgs) handleDefinitionPattern(elem, None)
+                            add(Assign, value, fromEnd = false)
+                    }
+
+                // single variable
+                case other@_ =>
+                    add(VariableDefinition, other, fromEnd = false)
+                    apply(other)
+                    maybeAddAndApply(optionalValue, Assign)
+            }
+        }
+
+        private def doMatch(tree: Tree): TR = {
+            tree match {
+                case Term.Do(body, condition) => TR(Some(DoWhile), Some(DoWhileEnd), traverse = _ => {
+                    encloseAndApply(body, TR(Some(DoBodyBegin), Some(DoBodyEnd)))
+                    apply(condition)
+                })
+                case Term.Assign(_) => TR(Some(Assign))
+                case Term.While(condition, body) => TR(Some(While), traverse = _ => {
+                    apply(condition)
+                    encloseAndApply(body, TR(Some(WhileBodyBegin), Some(WhileBodyEnd)))
+                })
+                case Term.For(enumerators, body) =>
+                    TR(Some(For), traverse = _ => {
+                        apply(enumerators)
+                        encloseAndApply(body, TR(Some(ForBodyBegin), Some(ForBodyEnd)))
+                    })
+                case Term.Try(expression, catchExpression, finallyExpression) => TR(Some(TryBegin),
+                    traverse = _ => {
+                        apply(expression)
+                        if (catchExpression.nonEmpty) {
+                            val start = catchExpression.head.pos
+                            val end = catchExpression.last.pos
+                            val length = end.endLine - start.startLine
+
+                            add(CatchBegin, start.startLine, start.startColumn, length)
+                            processCases(catchExpression)
+                            add(CatchEnd, end.endLine, end.endColumn, length)
+                        }
+
+                        maybeAddAndApply(finallyExpression, ScalaTokenConstants.Finally)
+                    })
+                case Term.TryWithHandler(expression, catchExpression, finallyExpression) => TR(Some(TryBegin),
+                    traverse = _ => {
+                        apply(expression)
+                        encloseAndApply(catchExpression, TR(Some(CatchBegin), Some(CatchEnd)))
+
+                        maybeAddAndApply(finallyExpression, ScalaTokenConstants.Finally)
+                    })
+                case Term.Apply(function, arguments) if !isStandardOperator(getMethodIdentifier(function)) && arguments.nonEmpty =>
+                    // function calls with no arguments are not covered here; see README
+                    TR(traverse = _ => {
+
+                        add(Apply, function, fromEnd = false)
+                        apply(function)
+                        for (argument <- arguments) {
+                            add(Argument, argument, fromEnd = false)
+                            argument match {
+                                case Term.Assign(assignee, value) =>
+                                    // default values
+                                    apply(assignee)
+                                    apply(value)
+                                case _ => apply(argument)
+                            }
+                        }
+                    })
+                case Term.NewAnonymous(_) => TR(Some(NewCreationBegin), Some(NewCreationEnd))
+                case Term.Return(_) => TR(Some(ScalaTokenConstants.Return))
+                case Term.Match(expression, cases) => TR(Some(MatchBegin), Some(MatchEnd), traverse = _ => {
+                    apply(expression)
+                    processCases(cases)
+                })
+                case Term.Throw(_) => TR(Some(Throw))
+                case Term.Function(_) => TR(Some(FunctionBegin), Some(FunctionEnd))
+                case Term.PartialFunction(cases) => TR(Some(PartialFunctionBegin), Some(PartialFunctionEnd), traverse = _ => {
+                    processCases(cases)
+                })
+                case Term.ForYield(enumerators, body) => TR(traverse = _ => {
+                    apply(enumerators)
+                    add(ForBodyBegin, body, fromEnd = false)
+                    encloseAndApply(body, TR(Some(Yield), Some(ForBodyEnd)))
+                })
+                case Term.If(condition, thenExpression, elseExpression) => TR(traverse = _ => {
+                    add(If, tree, fromEnd = false)
+                    apply(condition)
+
+                    encloseAndApply(thenExpression, TR(Some(IfBegin), Some(IfEnd)))
+
+                    elseExpression match {
+                        case Lit.Unit() => apply(elseExpression)
+                        case _ =>
+                            val elseStart = tree.pos.text.indexOf("else", thenExpression.pos.end - tree.pos.start)
+                            val elsePosition = Position.Range(tree.pos.input, tree.pos.start + elseStart, tree.pos.start + elseStart + 4)
+                            add(Else, elsePosition.startLine + 1, elsePosition.startColumn + 1, elsePosition.text.length)
+                            encloseAndApply(elseExpression, TR(Some(ElseBegin), Some(ElseEnd)))
+                    }
+                })
+
+                case scala.meta.Pkg(_) => TR(Some(Package))
+                case scala.meta.Import(_) => TR(Some(ScalaTokenConstants.Import))
+
+                case Defn.Def(modifiers, name, typeParameters, parameterLists, _, body) =>
+                    TR(traverse = _ => {
+                        applyRecursively(modifiers)
+                        add(MethodDef, name, fromEnd = false)
+                        assignRecursively(typeParameters, TypeParameter)
+                        assignRecursively(parameterLists, Parameter)
+
+                        encloseAndApply(body, TR(Some(MethodBegin), Some(MethodEnd)))
+                    })
+                case Defn.Macro(modifiers, macroName, typeParameters, parameterLists, declaredType, body) => TR(Some(Macro), traverse = _ => {
+                    applyRecursively(Seq(modifiers, typeParameters, parameterLists))
+                    encloseAndApply(body, TR(Some(MacroBegin), Some(MacroEnd)))
+                })
+                case Defn.Class(_) =>
+                    TR(Some(ClassBegin), Some(ClassEnd))
+                case Defn.Object(_) => TR(Some(ObjectBegin), Some(ObjectEnd))
+                case Defn.Trait(_) => TR(Some(TraitBegin), Some(TraitEnd))
+                case Defn.Type(_) => TR(Some(ScalaTokenConstants.Type))
+                case Defn.Var(modifiers, patterns, declaredType, optionalValue) => TR(traverse = _ => {
+                    apply(modifiers)
+                    for (pattern <- patterns) {
+                        handleDefinitionPattern(pattern, optionalValue)
+                    }
+                    apply(declaredType)
+                })
+                case Defn.Val(modifiers, patterns, declaredType, value) => TR(traverse = _ => {
+                    apply(modifiers)
+                    for (pattern <- patterns) {
+                        handleDefinitionPattern(pattern, Some(value))
+                    }
+                    apply(declaredType)
+
+                })
+
+                case Decl.Var(_) => TR(Some(VariableDefinition))
+                case Decl.Val(_) => TR(Some(VariableDefinition))
+                case Decl.Def(_) => TR(Some(MethodBegin), Some(MethodEnd))
+                case Decl.Type(_) => TR(Some(ScalaTokenConstants.Type))
+
+                case Ctor.Secondary(_) =>
+                    TR(Some(ConstructorBegin), Some(ConstructorEnd))
+
+                case Init(typeName, name, argumentLists) if argumentLists.nonEmpty => TR(traverse = _ => {
+                    assignRecursively(argumentLists, Argument, doApply = true)
+                })
+                case Enumerator.Guard(_) => TR(Some(Guard))
+
+                case Term.Param(_) => TR(traverse = _ => add(Parameter, tree, fromEnd = false))
+                case Term.ApplyInfix(_, operator, _, _) if operator.value.contains("=") && !Array("==", "!=").contains(operator.value) => TR(Some(Assign))
+                case Term.ApplyInfix(function, operator, typeArgs, arguments) if !isStandardOperator(operator.value) => TR(traverse = _ => {
+                    add(Apply, tree, fromEnd = false)
+                    apply(function)
+                    assignRecursively(typeArgs, TypeArgument, doApply = true)
+                    assignRecursively(arguments, Argument, doApply = true)
+                })
+                case Term.Select(refObj, member) =>
+                    TR(traverse = _ => {
+                        apply(refObj)
+                        if (!isStandardOperator(member.value)) add(ScalaTokenConstants.Member, member, fromEnd = false)
+                        apply(member)
+                    })
+                case Term.ApplyType(_, typeArgs) => TR(traverse = _ => {
+                    add(Apply, tree, fromEnd = false)
+                    assignRecursively(typeArgs, TypeArgument)
+                })
+                case Term.New(_) => TR(Some(NewObject))
+                case Self(_) => TR(Some(SelfType))
+                case block@Term.Block(_) => block.parent match {
+                    // inner block
+                    case Some(Term.Apply(_)) => TR(Some(BlockStart), Some(BlockEnd))
+                    // block in an expression context, e.g. for, if, while
+                    case _ => TR()
+                }
+                case Enumerator.Generator(_) => TR(Some(EnumGenerator))
+                case meta.Type.Param(_) => TR(Some(TypeParameter))
+
+                case _ => TR()
+            }
+        }
+
+        /**
+         * Traverses a sequence of elements, which may in turn be lists as well.
+         *
+         * @param elements a sequence of values, Trees or Lists
+         * @tparam T the type of elements
+         */
+        private def applyRecursively[T](elements: Seq[T]): Unit = elements.foreach {
+            case tree: Tree => apply(tree)
+            case treeList: List[_] => applyRecursively(treeList)
+            case _ =>
+        }
+
+        override def apply(tree: Tree): Unit = {
+            val record: TR = doMatch(tree)
+
+            record.before match {
+                case Some(value) => add(value, tree, fromEnd = false)
+                case None =>
+            }
+
+            record.traverse(tree)
+
+            record.after match {
+                case Some(value) => add(value, tree, fromEnd = true)
+                case None =>
+            }
+        }
+
+        /**
+         * Assigns the same token type to each element of a list of e.g. parameters.
+         *
+         * @param els       list of equal elements
+         * @param tokenType type of token to assign to each of the elements
+         * @param doApply   if true, the substructure of each element is traversed
+         * @tparam T list may contain more lists or leaves
+         */
+        private def assignRecursively[T](els: List[T], tokenType: ScalaTokenConstants.Value, doApply: Boolean = false): Unit = els.foreach {
+            case treeList: List[_] => assignRecursively(treeList, tokenType)
+            case el: Tree =>
+                add(tokenType, el, fromEnd = false)
+                if (doApply) apply(el)
+        }
+
+        /**
+         * Adds a token to the beginning of the tree, traverses the tree, adds a token at the end of the tree.
+         * The token types are given by record.
+         *
+         * @param tree   Tree to enclose and traverse
+         * @param record contains token types
+         */
+        private def encloseAndApply(tree: Tree, record: TR): Unit = {
+            record.before match {
+                case Some(value) => add(value, tree, fromEnd = false)
+                case None =>
+            }
+
+            apply(tree)
+
+            record.after match {
+                case Some(value) => add(value, tree, fromEnd = true)
+                case None =>
+            }
+        }
+    }
+
+    def parse(directory: File, files: Array[String]): TokenList = {
+        tokens = new TokenList
+        errors = 0
+
+        for (file <- files) {
+            currentFile = file
+            if (!parseFile(directory, file)) {
+                errors += 1
+            }
+            System.gc()
+        }
+
+        tokens
+    }
+
+    private def parseFile(directory: File, fileName: String): Boolean = {
+        currentFile = fileName
+
+        try {
+            val file = new File(directory, fileName)
+            val bytes = java.nio.file.Files.readAllBytes(file.toPath)
+            val text = new String(bytes, "UTF-8")
+            val input = Input.VirtualFile(file.getPath, text)
+            val ast = input.parse[Source].get
+            traverser(ast)
+
+            add(FileEnd, text.count(_ == '\n') - 1, 0, 0)
+        } catch {
+            case exception: Throwable =>
+                exception.printStackTrace()
+                return false
+        }
+
+        true
+    }
+
+    /**
+     * Adds a token to the token list.
+     *
+     * @param tokenType the type of the token
+     * @param line      line of the occurrence in the file
+     * @param column    column of the occurrence in the file
+     * @param length    length of the occurrence in the file
+     */
+    private def add(tokenType: ScalaTokenConstants.Value, line: Int, column: Int, length: Int): Unit = {
+        tokens.addToken(new ScalaToken(tokenType.id, currentFile, line, column, length))
+    }
+
+
+    /**
+     * Adds a token to the token list.
+     *
+     * @param tokenType the type of the token
+     * @param node      the tree that marks the occurrence
+     * @param fromEnd   if true, the token is added at the end of the tree (length 0).
+     */
+    private def add(tokenType: ScalaTokenConstants.Value, node: Tree, fromEnd: Boolean): Unit = {
+        if (node.pos.text.nonEmpty) {
+            // SELF type tokens with no text content mess up the sequence
+            tokens.addToken(new ScalaToken(tokenType, currentFile, node.pos, fromEnd))
+        }
+    }
+
+}

jplag.frontend.scala/src/main/scala/de/jplag/scala/ScalaToken.scala

@@ -0,0 +1,36 @@
+package de.jplag.scala
+
+import de.jplag.Token
+
+import scala.meta.inputs.Position
+
+class ScalaToken(tokenType: Int,
+                 file: java.lang.String,
+                 var line: Int,
+                 var column: Int,
+                 var length: Int)
+  extends Token(tokenType, file, line, column, length) {
+
+    def this(tokenType: ScalaTokenConstants.Value, file: String, position: Position, fromEnd: Boolean) {
+        this(
+            tokenType.id,
+            file,
+            (if (fromEnd) position.endLine else position.startLine) + 1,
+            (if (fromEnd) position.endColumn else position.startColumn) + 1,
+            if (fromEnd) 0 else position.text.length,
+        )
+    }
+
+    override def toString: String = {
+        try {
+            ScalaTokenConstants(`type`).toString
+        } catch {
+            case _: Throwable => "Unknown!"
+        }
+    }
+
+    /**
+     * @return a string representation depending on the type of the token.
+     */
+    override protected def type2string(): String = ScalaTokenConstants.apply(tokenType).toString
+}

jplag.frontend.scala/src/main/scala/de/jplag/scala/ScalaTokenConstants.scala

@@ -0,0 +1,83 @@
+package de.jplag.scala
+
+import de.jplag.TokenConstants
+
+/**
+ * This Java-compatible enumeration stores the possible types of tokens.
+ */
+object ScalaTokenConstants extends Enumeration with TokenConstants {
+
+  val FileEnd: Value = Value(TokenConstants.FILE_END, "<EOF>")
+  val SeparatorToken: Value = Value(TokenConstants.SEPARATOR_TOKEN, "--------")
+  val Package: Value = Value("PACKAGE")
+  val Import: Value = Value("IMPORT")
+  val ClassBegin: Value = Value("CLASS{")
+  val ClassEnd: Value = Value("}CLASS")
+  val MethodDef: Value = Value("METHOD")
+  val MethodBegin: Value = Value("METHOD{")
+  val MethodEnd: Value = Value("}METHOD")
+  val VariableDefinition: Value = Value("VAR_DEF")
+  val DoWhile: Value = Value("DO-WHILE")
+  val DoWhileEnd: Value = Value("END-DO-WHILE")
+  val DoBodyBegin: Value = Value("DO{")
+  val DoBodyEnd: Value = Value("}DO")
+  val While: Value = Value("WHILE")
+  val WhileBodyBegin: Value = Value("WHILE{")
+  val WhileBodyEnd: Value = Value("}WHILE")
+  val For: Value = Value("FOR")
+  val ForBodyBegin: Value = Value("FOR{")
+  val ForBodyEnd: Value = Value("}FOR")
+  val CaseStatement: Value = Value("CASE")
+  val CaseBegin: Value = Value("CASE{")
+  val CaseEnd: Value = Value("}CASE")
+  val TryBegin: Value = Value("TRY{")
+  val CatchBegin: Value = Value("CATCH{")
+  val CatchEnd: Value = Value("}CATCH")
+  val Finally: Value = Value("FINALLY")
+  val If: Value = Value("IF")
+  val IfBegin: Value = Value("IF{")
+  val IfEnd: Value = Value("}IF")
+  val Else: Value = Value("ELSE")
+  val ElseBegin: Value = Value("ELSE{")
+  val ElseEnd: Value = Value("}ELSE")
+  val Return: Value = Value("RETURN")
+  val Throw: Value = Value("THROW")
+  val NewCreationBegin: Value = Value("NEW{")
+  val NewCreationEnd: Value = Value("}NEW")
+  val Apply: Value = Value("APPLY")
+  val Assign: Value = Value("ASSIGN")
+  val TraitBegin: Value = Value("TRAIT{")
+  val TraitEnd: Value = Value("}TRAIT")
+  val ConstructorBegin: Value = Value("CONSTR{")
+  val ConstructorEnd: Value = Value("}CONSTR")
+  val MatchBegin: Value = Value("MATCH{")
+  val MatchEnd: Value = Value("}MATCH")
+  val Guard: Value = Value("GUARD")
+  val ObjectBegin: Value = Value("OBJECT{")
+  val ObjectEnd: Value = Value("}OBJECT")
+  val Macro: Value = Value("MACRO")
+  val MacroBegin: Value = Value("MACRO{")
+  val MacroEnd: Value = Value("}MACRO")
+  val Type: Value = Value("TYPE")
+
+  val FunctionBegin: Value = Value("FUNC{")
+  val FunctionEnd: Value = Value("}FUNC")
+  val PartialFunctionBegin: Value = Value("PFUNC{")
+  val PartialFunctionEnd: Value = Value("}PFUNC")
+
+  val Yield: Value = Value("YIELD")
+
+  val Parameter: Value = Value("PARAM")
+  val Argument: Value = Value("ARG")
+  val NewObject: Value = Value("NEW()")
+  val SelfType: Value = Value("SELF")
+  val TypeParameter: Value = Value("T_PARAM")
+  val TypeArgument: Value = Value("T_ARG")
+  val BlockStart: Value = Value("{")
+  val BlockEnd: Value = Value("}")
+  val EnumGenerator: Value = Value("ENUMERATE")
+  val Member: Value = Value("MEMBER")
+
+  def numberOfTokens() : Int = maxId
+}
+

jplag.frontend.scala/src/test/java/de/jplag/scala/ScalaFrontendTest.java

@@ -0,0 +1,182 @@
+package de.jplag.scala;
+
+import static org.junit.jupiter.api.Assertions.fail;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.stream.Collectors;
+import java.util.stream.IntStream;
+import java.util.stream.StreamSupport;
+
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import de.jplag.Token;
+import de.jplag.TokenConstants;
+import de.jplag.TokenList;
+import de.jplag.TokenPrinter;
+
+class ScalaFrontendTest {
+
+    /**
+     * Test source file that is supposed to produce a complete set of tokens, i.e. all types of tokens.
+     */
+    private static final String COMPLETE_TEST_FILE = "Complete.scala";
+
+    /**
+     * Regular expression that describes lines consisting only of whitespace and optionally a line comment.
+     */
+    private static final String EMPTY_OR_SINGLE_LINE_COMMENT = "\\s*(//.*|/\\*.*\\*/)?";
+
+    /**
+     * Regular expression that describes lines containing the start of a multiline comment and no code before it.
+     */
+    private static final String DELIMITED_COMMENT_START = "\\s*/\\*(?:(?!\\*/).)*$";
+
+    /**
+     * Regular expression that describes lines containing the end of a multiline comment and no more code after that.
+     */
+    private static final String DELIMITED_COMMENT_END = ".*\\*/\\s*$";
+    private static final String EMPTY_STRING = "";
+    private static final int NOT_SET = -1;
+    private static final double EPSILON = 1E-6;
+
+    private final Logger logger = LoggerFactory.getLogger("Scala frontend test");
+    private final String[] testFiles = new String[] {"Parser.scala", COMPLETE_TEST_FILE};
+    private final File testFileLocation = Path.of("src", "test", "resources", "de", "jplag", "scala").toFile();
+    private Language language;
+
+    @BeforeEach
+    void setup() {
+        language = new Language();
+    }
+
+    @Test
+    void parseTestFiles() {
+        for (String fileName : testFiles) {
+            TokenList tokens = language.parse(testFileLocation, new String[] {fileName});
+            String output = TokenPrinter.printTokens(tokens, testFileLocation, List.of(fileName));
+            logger.info(output);
+
+            if (fileName.equals(COMPLETE_TEST_FILE)) {
+                testTokenCoverage(tokens, fileName);
+            }
+            testSourceCoverage(fileName, tokens);
+
+        }
+    }
+
+    /**
+     * Confirms that every type of ScalaToken has a Sting representation associated to it.
+     */
+    @Test
+    void testTokenToString() {
+        var missingTokens = IntStream.range(0, ScalaTokenConstants.numberOfTokens())
+                .mapToObj(type -> new ScalaToken(type, EMPTY_STRING, NOT_SET, NOT_SET, NOT_SET))
+                .filter(token -> token.type2string().contains("UNKNOWN")).toList();
+
+        if (!missingTokens.isEmpty()) {
+            var typeList = missingTokens.stream().map(Token::getType).map(Object::toString).collect(Collectors.joining(", "));
+            fail("Found token types with no string representation: %s".formatted(typeList));
+        }
+
+    }
+
+    /**
+     * Confirms that the code is covered to a basic extent, i.e. each line of code contains at least one token.
+     * @param fileName a code sample file name
+     * @param tokens the TokenList generated from the sample
+     */
+    private void testSourceCoverage(String fileName, TokenList tokens) {
+        var testFile = new File(testFileLocation, fileName);
+
+        try {
+            var lines = Files.readAllLines(testFile.toPath());
+
+            // All lines that contain code
+            var codeLines = new ArrayList<>(getCodeLines(lines));
+            // All lines that contain token
+            var tokenLines = IntStream.range(0, tokens.size()).mapToObj(tokens::getToken).mapToInt(Token::getLine).distinct().boxed().toList();
+
+            // Keep only lines that have no tokens
+            codeLines.removeAll(tokenLines);
+
+            var coverage = 1.d - (codeLines.size() * 1.d / (codeLines.size() + tokenLines.size()));
+            if (coverage == 1) {
+                logger.info("All lines covered.");
+            } else {
+                logger.info("Coverage: %.1f%%.".formatted(coverage * 100));
+                logger.info("Missing lines {}", codeLines);
+                if (coverage - 0.9 <= EPSILON) {
+                    fail("Source coverage is unsatisfactory");
+                }
+            }
+
+        } catch (IOException exception) {
+            logger.info("Error while reading test file %s".formatted(fileName), exception);
+            fail();
+        }
+    }
+
+    /**
+     * Gets the line numbers of lines containing actual code, omitting empty lines and comment lines.
+     * @param lines lines of a code file
+     * @return an array of the line numbers of code lines
+     */
+    private List<Integer> getCodeLines(List<String> lines) {
+        // This boxed boolean can be accessed from within the lambda method below
+        var state = new Object() {
+            boolean insideComment = false;
+        };
+
+        var codeLines = IntStream.rangeClosed(1, lines.size()).sequential().filter(idx -> {
+            String line = lines.get(idx - 1);
+            if (line.matches(EMPTY_OR_SINGLE_LINE_COMMENT)) {
+                return false;
+            } else if (line.matches(DELIMITED_COMMENT_START)) {
+                state.insideComment = true;
+                return false;
+            } else if (state.insideComment) {
+                // This fails if code follows after '*/'. If the code is formatted well, this should not happen.
+                if (line.matches(DELIMITED_COMMENT_END)) {
+                    state.insideComment = false;
+                }
+                return false;
+            }
+            return true;
+        }).boxed().toList();
+
+        return codeLines;
+
+    }
+
+    /**
+     * Confirms that all Token types are 'reachable' with a complete code example.
+     * @param tokens TokenList which is supposed to contain all types of tokens
+     * @param fileName The file name of the complete code example
+     */
+    private void testTokenCoverage(TokenList tokens, String fileName) {
+        var foundTokens = StreamSupport.stream(tokens.allTokens().spliterator(), true).mapToInt(Token::getType).distinct().boxed().toList();
+        var allTokens = IntStream.range(0, ScalaTokenConstants.numberOfTokens()).boxed().toList();
+        allTokens = new ArrayList<>(allTokens);
+
+        // Only non-found tokens are left
+        allTokens.removeAll(foundTokens);
+        // Exclude SEPARATOR_TOKEN, as it does not occur
+        allTokens.remove((Integer) (TokenConstants.SEPARATOR_TOKEN));
+
+        if (!allTokens.isEmpty()) {
+            var notFoundTypes = allTokens.stream().map(ScalaTokenConstants::apply).toList();
+            fail("Some %d token types were not found in the complete code example '%s':\n%s".formatted(notFoundTypes.size(), fileName,
+                    notFoundTypes));
+        }
+
+    }
+
+}

jplag.frontend.scala/src/test/resources/de/jplag/scala/Complete.scala

@@ -0,0 +1,162 @@
+package mypackage
+
+import scala.annotation.tailrec
+import scala.otherpackage
+
+class Complete extends SuperClass with MyTrait {
+
+    val constantMember = 1
+
+    def method(arg: ArgType): Unit = {
+
+        var variable : Int = 2
+
+        do method() while (false)
+        while (false) method()
+
+        for (index <- 0 to 10 by 1 if isPrime(index)) {
+            method(index)
+            (index % 2) match {
+                case 0 => "even"
+                case 1 => "odd"
+            }
+        }
+
+        if (true) {
+
+        } else {
+
+        }
+
+        try {
+            method()
+            throw new Exception()
+        } catch {
+            case exception : Exception if true =>
+            case _ =>
+        } finally {
+
+        }
+
+        val anonymousClassObject = new AnyVal {}
+
+    }
+
+    def getOne() : Int = {
+        return 1
+    }
+
+    object NumberGenerator {
+
+        def getNumber(i : Int): Unit = {
+            if (i <= 0) i else 1 + getNumber(i - 1)
+        }
+
+    }
+
+    class Person(var firstName: String, var lastName: String) {
+
+        def this(var firstName: String) = {
+            this(firstName, "Smith")
+        }
+    }
+
+    def create(): Unit = {
+        val obj = new Person("Mickey", "Mouse")
+    }
+
+    def applyPartialFunction(strings: Seq[String]) =
+        strings.collect({
+            case s if s.length > 5 => s
+            case _ =>
+        })
+
+    def applyTotalFunction(strings: Seq[String]) =
+        strings.find(s => {
+            s.substring(0,3).equals("abc")
+        })
+
+    def callMacro(msg: Any) = macro macroImpl
+
+    def macroImpl(c: Context)(msg: c.Expr[Any]) = println(msg)
+
+    def yieldingMethod(list: Seq[String]) = {
+        for (string <- list if string.startsWith("_")) yield string.substring(1)
+    }
+
+    trait MyTrait[U] {
+        self =>
+        type T <: U
+        val member: T
+    }
+
+    class MyTraitImpl extends MyTrait[Complete] {}
+
+    def reverse[T](list: Seq[T]): Seq[T] = {
+        if (list.isEmpty) list
+        else reverse(list.tail) :: list.head
+    }
+
+    def useReverse(): Unit = {
+        val list = Array[Int](3, 2, 1)
+        reverse[Int](list)
+    }
+
+    def innerBlock(): Unit = {
+        val a = 1
+        {
+            val b = 2
+        }
+        println("b is not accessible here.")
+    }
+
+    def operators(): Unit = {
+        // This is not treated as a function call
+        val a = 1 + 2
+        // This is treated as a function call
+        val b = (1).+(2)
+    }
+
+    def functionObject(): Unit = {
+        var f : Int => Int = _ * 2
+        f(2)
+
+        ((i: Int) => i * 2)(2)
+    }
+
+    def memberChain(): Unit = {
+        a.b.c
+    }
+
+    def infix() : Unit = {
+        val list = List(1,2,3)
+        val list2 = list reverse_::: list
+    }
+
+    def power(base: Int, exponent: Int): Int = {
+        if (exponent == 0) 1
+        else if (exponent == 1) base
+        else if (exponent % 2 == 0) {
+            ((i: Int) => i * i) (power(base, exponent / 2))
+        }
+        else base * power(base, exponent - 1)
+    }
+
+    def methodCall() : Unit = {
+        myObject.member // may be member reference or method call
+        // gets MEMBER token
+
+        myObject.member2() // must be method call
+        // gets MEMBER token
+
+        myObject.member3(arg1, arg2) // must be method call
+        // gets APPLY MEMBER ARG ARG tokens
+    }
+
+    /*
+    Enums are a feature of Scala3, not yet supported by the parser.
+    enum Menu:
+        case PizzaMargharita, SpaghettiBolognese, Fries
+    */
+
+}

jplag.frontend.scala/src/test/resources/de/jplag/scala/Parser.scala

@@ -0,0 +1,376 @@
+package de.jplag.scala
+
+import de.jplag.scala.ScalaTokenConstants._
+import de.jplag.{AbstractParser, TokenList}
+
+import java.io.File
+import scala.meta._
+
+
+class Parser extends AbstractParser {
+    private var currentFile: String = _
+
+    private var tokens: TokenList = _
+
+    private val traverser: Traverser = new Traverser {
+
+        private val Operators : Array[String] = Array(
+            "+", "-", "*", "/", "%", "**",
+            "==", "!=", ">", "<", ">=", "<=",
+            "&&", "||", "!",
+            "=", "+=", "-=", "*=", "/=", "%=", "**=", "<<=", ">>=", ">>>=", "&=", "^=", "|=",
+            "&", "|", "^", "<<", ">>", "~", ">>>",
+            "++", "::", ":::",
+            "<:", ">:", "#"
+        )
+
+        /**
+         * TraverserRecord; Identifier kept short for readability
+         * @param before opt. token type to insert at the beginning of the tree
+         * @param after opt. token type to insert at the end of the tree
+         * @param traverse custom method of traversing the tree
+         */
+        case class TR(before: Option[ScalaTokenConstants.Value] = None,
+                      after: Option[ScalaTokenConstants.Value] = None,
+                      traverse: Tree => Unit = _.children.foreach(traverser.apply)
+                     )
+
+        /**
+         * Adds a token at the beginning of the tree and traverses it if a tree is given.
+         * @param tree opt. tree
+         * @param token token type to insert at the beginning of the tree
+         */
+        private def maybeAddAndApply(tree: Option[Tree], token: ScalaTokenConstants.Value): Unit = tree match {
+            case Some(exp) =>
+                add(token, exp, fromEnd = false)
+                apply(exp)
+            case None =>
+        }
+
+        private def processCases(cases: List[Case]): Unit = cases.foreach {
+            case c@Case(pattern, condition, body) =>
+                add(ScalaTokenConstants.CaseStatement, c, fromEnd = false)
+                applyRecursively(Seq(pattern, condition))
+
+                encloseAndApply(body, TR(Some(CaseBegin), Some(CaseEnd)))
+            case _ =>
+        }
+
+        private def isStandardOperator(op: String): Boolean = {
+            Operators.contains(op)
+        }
+
+        private def getMethodIdentifier(fun: Term): String = {
+            fun.toString().split("\\.").last
+        }
+
+        private def doMatch(tree: Tree): TR = {
+            tree match {
+                case Term.Do(body, condition) => TR(Some(DoWhile), Some(DoWhileEnd), traverse = _ => {
+                    encloseAndApply(body, TR(Some(DoBodyBegin), Some(DoBodyEnd)))
+                    apply(condition)
+                })
+                case Term.Assign(_) => TR(Some(Assign))
+                case Term.While(condition, body) => TR(Some(While), traverse = _ => {
+                    apply(condition)
+                    encloseAndApply(body, TR(Some(WhileBodyBegin), Some(WhileBodyEnd)))
+                })
+                case Term.For(enumerators, body) =>
+                    TR(Some(For), traverse = _ => {
+                        apply(enumerators)
+                        encloseAndApply(body, TR(Some(ForBodyBegin), Some(ForBodyEnd)))
+                    })
+                case Term.Try(expr, catchExpr, finallyExpr) => TR(Some(TryBegin),
+                    traverse = _ => {
+                        apply(expr)
+                        if (catchExpr.nonEmpty) {
+                            val start = catchExpr.head.pos
+                            val end = catchExpr.last.pos
+                            val len = end.endLine - start.startLine
+
+                            add(CatchBegin, start.startLine, start.startColumn, len)
+                            processCases(catchExpr)
+                            add(CatchEnd, end.endLine, end.endColumn, len)
+                        }
+
+                        maybeAddAndApply(finallyExpr, ScalaTokenConstants.Finally)
+                    })
+                case Term.TryWithHandler(expr, catchp, finallyp) => TR(Some(TryBegin),
+                    traverse = _ => {
+                        apply(expr)
+                        encloseAndApply(catchp, TR(Some(CatchBegin), Some(CatchEnd)))
+
+                        maybeAddAndApply(finallyp, ScalaTokenConstants.Finally)
+                    })
+                case Term.Apply(fun, args) if !isStandardOperator(getMethodIdentifier(fun)) =>
+                    // `f()` can also be written as `f`, so we simply don't see `f()` as a function call
+                    // But Java style function calls with no args should still be recognized
+                    TR(traverse = _ => {
+
+                        add(Apply, fun, fromEnd = false)
+                        apply(fun)
+                        for (arg <- args) {
+                            add(Argument, arg, fromEnd = false)
+                            arg match {
+                                case Term.Assign(lhs, rhs) =>
+                                    apply(lhs)
+                                    apply(rhs)
+                                case _ => apply(arg)
+                            }
+                        }
+                    })
+                case Term.NewAnonymous(_) => TR(Some(NewCreationBegin), Some(NewCreationEnd))
+                case Term.Return(_) => TR(Some(ScalaTokenConstants.Return))
+                case Term.Match(expr, cases) => TR(Some(MatchBegin), Some(MatchEnd), traverse = _ => {
+                    apply(expr)
+                    processCases(cases)
+                })
+                case Term.Throw(_) => TR(Some(Throw))
+                case Term.Function(_) => TR(Some(FunctionBegin), Some(FunctionEnd))
+                case Term.PartialFunction(cases) => TR(Some(PartialFunctionBegin), Some(PartialFunctionEnd), traverse = _ => {
+                    processCases(cases)
+                })
+                case Term.ForYield(enumerators, body) => TR(traverse = _ => {
+                    apply(enumerators)
+                    add(ForBodyBegin, body, fromEnd = false)
+                    encloseAndApply(body, TR(Some(Yield), Some(ForBodyEnd)))
+                })
+                case Term.If(conditionExpr, thenExpression, elseExpression) => TR(traverse = _ => {
+                    add(If, tree, fromEnd = false)
+                    apply(conditionExpr)
+
+                    encloseAndApply(thenExpression, TR(Some(IfBegin), Some(IfEnd)))
+
+                    elseExpression match {
+                        case Lit.Unit() => apply(elseExpression)
+                        case _ =>
+                            val elseStart = tree.pos.text.indexOf("else", thenExpression.pos.end - tree.pos.start)
+                            val elsePosition = Position.Range(tree.pos.input, tree.pos.start + elseStart, tree.pos.start + elseStart + 4)
+                            add(Else, elsePosition.startLine + 1, elsePosition.startColumn + 1, elsePosition.text.length)
+                            encloseAndApply(elseExpression, TR(Some(ElseBegin), Some(ElseEnd)))
+                    }
+                })
+
+                case scala.meta.Pkg(_) => TR(Some(Package))
+                case scala.meta.Import(_) => TR(Some(ScalaTokenConstants.Import))
+
+                case Defn.Def(mods, name, typeParams, paramss, _, body) =>
+                    TR(traverse = _ => {
+                        applyRecursively(mods)
+                        add(MethodDef, name, fromEnd = false)
+                        assignRecursively(typeParams, TypeParameter)
+                        assignRecursively(paramss, Parameter)
+
+                        encloseAndApply(body, TR(Some(MethodBegin), Some(MethodEnd)))
+                    })
+                case Defn.Macro(mods, name, tparams, paramss, declType, body) => TR(Some(Macro), traverse = _ => {
+                    applyRecursively(Seq(mods, tparams, paramss))
+                    encloseAndApply(body, TR(Some(MacroBegin), Some(MacroEnd)))
+                })
+                case Defn.Class(_) =>
+                    TR(Some(ClassBegin), Some(ClassEnd))
+                case Defn.Object(_) => TR(Some(ObjectBegin), Some(ObjectEnd))
+                case Defn.Trait(_) => TR(Some(TraitBegin), Some(TraitEnd))
+                case Defn.Type(_) => TR(Some(ScalaTokenConstants.Type))
+                case Defn.Var(mods, pats, decltype, rhs) => TR(traverse = _ => {
+                    apply(mods)
+                    for (pat <- pats) {
+                        add(VariableDefinition, pat, fromEnd = false)
+                        apply(pat)
+                        apply(decltype)
+
+                        rhs match {
+                            case Some(realRhs) =>
+                                add(Assign, realRhs, fromEnd = false)
+                                apply(realRhs)
+                            case None =>
+                        }
+                    }
+                })
+                case Defn.Val(mods, pats, decltype, rhs) => TR(traverse = _ => {
+                    apply(mods)
+                    for (pat <- pats) {
+                        add(VariableDefinition, pat, fromEnd = false)
+                        apply(pat)
+                        apply(decltype)
+
+                        add(Assign, rhs, fromEnd = false)
+                        apply(rhs)
+                    }
+
+                })
+
+                case Decl.Var(_) => TR(Some(VariableDefinition))
+                case Decl.Val(_) => TR(Some(VariableDefinition))
+                case Decl.Def(_) => TR(Some(MethodBegin), Some(MethodEnd))
+                case Decl.Type(_) => TR(Some(ScalaTokenConstants.Type))
+
+                case Ctor.Secondary(_) =>
+                    TR(Some(ConstructorBegin), Some(ConstructorEnd))
+
+                case Init(tpe, name, argss) if argss.nonEmpty => TR(traverse = _ => {
+                    assignRecursively(argss, Argument, doApply = true)
+                })
+                case Enumerator.Guard(_) => TR(Some(Guard))
+
+                case Term.Param(_) => TR(traverse = _ => add(Parameter, tree, fromEnd = false))
+                case Term.NewAnonymous(_) => TR(Some(ClassBegin), Some(ClassEnd))
+                case Term.ApplyInfix(_, op, _, _) if op.value.contains("=") && !Array("==", "!=").contains(op.value) => TR(Some(Assign))
+                case Term.ApplyInfix(fun, op, typeArgs, args) if !isStandardOperator(op.value) => TR(traverse = _ => {
+                    add(Apply, tree, fromEnd = false)
+                    apply(fun)
+                    assignRecursively(typeArgs, TypeArgument, doApply = true)
+                    assignRecursively(args, Argument, doApply = true)
+                })
+                case Term.Select(refObj, member) =>
+                    TR(traverse = _ => {
+                        apply(refObj)
+                        if (!isStandardOperator(member.value)) add(ScalaTokenConstants.Member, member, fromEnd = false)
+                        apply(member)
+                    })
+                case Term.ApplyType(_, typeArgs) => TR(traverse = _ => {
+                    add(Apply, tree, fromEnd = false)
+                    assignRecursively(typeArgs, TypeArgument)
+                })
+                case Term.New(_) => TR(Some(NewObject))
+                case Self(_) => TR(Some(SelfType))
+                case block@Term.Block(_) => block.parent match {
+                        // inner block
+                        case Some(Term.Apply(_)) => TR(Some(BlockStart), Some(BlockEnd))
+                        // block in an expression context, e.g. for, if, while
+                        case _ => TR()
+                    }
+                case Enumerator.Generator(_) => TR(Some(EnumGenerator))
+                case meta.Type.Param(_) => TR(Some(TypeParameter))
+
+                case _ => TR()
+            }
+        }
+
+        /**
+         * Traverses a sequence of elements, which may in turn be lists as well.
+         * @param els a sequence of values, Trees or Lists
+         * @tparam T the type of elements
+         */
+        def applyRecursively[T](els: Seq[T]): Unit = els.foreach {
+            case tree: Tree => apply(tree)
+            case treeList: List[_] => applyRecursively(treeList)
+            case _ =>
+        }
+
+        override def apply(tree: Tree): Unit = {
+            val record: TR = doMatch(tree)
+
+            record.before match {
+                case Some(value) => add(value, tree, fromEnd = false)
+                case None =>
+            }
+
+            record.traverse(tree)
+
+            record.after match {
+                case Some(value) => add(value, tree, fromEnd = true)
+                case None =>
+            }
+        }
+
+        /**
+         * Assigns the same token type to each element of a list of e.g. parameters.
+         * @param els list of equal elements
+         * @param tokenType type of token to assign to each of the elements
+         *                   @param doApply if true, the substructure of each element is traversed
+         * @tparam T list may contain more lists or leaves
+         */
+        def assignRecursively[T](els: List[T], tokenType: ScalaTokenConstants.Value, doApply: Boolean = false): Unit = els.foreach {
+            case treeList: List[_] => assignRecursively(treeList, tokenType)
+            case el: Tree =>
+                add(tokenType, el, fromEnd = false)
+                if (doApply) apply(el)
+        }
+
+        /**
+         * Adds a token to the beginning of the tree, traverses the tree, adds a token at the end of the tree.
+         * The token types are given by record.
+         *
+         * @param tree Tree to enclose and traverse
+         * @param record contains token types
+         */
+        def encloseAndApply(tree: Tree, record: TR): Unit = {
+            record.before match {
+                case Some(value) => add(value, tree, fromEnd = false)
+                case None =>
+            }
+
+            apply(tree)
+
+            record.after match {
+                case Some(value) => add(value, tree, fromEnd = true)
+                case None =>
+            }
+        }
+    }
+
+
+    def parse(dir: File, files: Array[String]): TokenList = {
+        tokens = new TokenList
+        errors = 0
+
+        for (file <- files) {
+            currentFile = file
+            // getProgram.print(null, "Parsing file" + file + "\n")
+            if (!parseFile(dir, file)) {
+                errors += 1
+            }
+            System.gc()
+        }
+
+        tokens
+    }
+
+    private def parseFile(dir: File, file: String): Boolean = {
+        currentFile = file
+
+        try {
+            val f = new File(dir, file)
+            val bytes = java.nio.file.Files.readAllBytes(f.toPath)
+            val text = new String(bytes, "UTF-8")
+            val input = Input.VirtualFile(f.getPath, text)
+            val ast = input.parse[Source].get
+            traverser(ast)
+
+            add(FileEnd, text.count(_ == '\n') - 1, 0, 0)
+        } catch {
+            case e: Throwable =>
+                e.printStackTrace()
+                return false
+        }
+
+        true
+    }
+
+    /**
+     * Adds a token to the token list.
+     * @param tType the type of the token
+     * @param line line of the occurrence in the file
+     * @param column column of the occurrence in the file
+     * @param length length of the occurrence in the file
+     */
+    private def add(tType: ScalaTokenConstants.Value, line: Int, column: Int, length: Int): Unit = {
+        tokens.addToken(new ScalaToken(tType.id, currentFile, line, column, length))
+    }
+
+
+    /**
+     * Adds a token to the token list.
+     * @param tType the type of the token
+     * @param node the tree that marks the occurrence
+     * @param fromEnd if true, the token is added at the end of the tree (length 0).
+     */
+    private def add(tType: ScalaTokenConstants.Value, node: Tree, fromEnd: Boolean): Unit = {
+        if (node.pos.text.nonEmpty) {
+            // SELF type tokens with no text content mess up the sequence
+            tokens.addToken(new ScalaToken(tType, currentFile, node.pos, fromEnd))
+        }
+    }
+
+}

jplag/pom.xml

@@ -57,6 +57,10 @@
             <groupId>de.jplag</groupId>
             <artifactId>rlang</artifactId>
         </dependency>
+        <dependency>
+            <groupId>de.jplag</groupId>
+            <artifactId>scala</artifactId>
+        </dependency>
         <dependency>
             <groupId>de.jplag</groupId>
             <artifactId>scheme</artifactId>

jplag/src/main/java/de/jplag/options/LanguageOption.java

@@ -17,6 +17,7 @@ public enum LanguageOption {
     KOTLIN("kotlin", "de.jplag.kotlin.Language"),
     R_LANG("rlang", "de.jplag.rlang.Language"),
     RUST("rust", "de.jplag.rust.Language"),
+    SCALA("scala", "de.jplag.scala.Language"),
     CHAR("char", "de.jplag.chars.Language"),
     TEXT("text", "de.jplag.text.Language"),
     SCHEME("scheme", "de.jplag.scheme.Language");

pom.xml

@@ -76,6 +76,7 @@
         <maven.compiler.target>17</maven.compiler.target>
         <spotless.version>2.24.0</spotless.version>
         <slf4j.version>1.7.36</slf4j.version>
+        <junit.version>5.9.0</junit.version>
 
         <antlr2.version>2.7.7</antlr2.version>
         <antlr3.version>3.5</antlr3.version>
@@ -177,6 +178,11 @@
                 <artifactId>rust</artifactId>
                 <version>${revision}</version>
             </dependency>
+            <dependency>
+                <groupId>de.jplag</groupId>
+                <artifactId>scala</artifactId>
+                <version>${revision}</version>
+            </dependency>
             <dependency>
                 <groupId>de.jplag</groupId>
                 <artifactId>scheme</artifactId>
@@ -197,6 +203,11 @@
                 <artifactId>frontend-testutils</artifactId>
                 <version>${revision}</version>
             </dependency>
+            <dependency>
+                <groupId>de.jplag</groupId>
+                <artifactId>endToEndTesting</artifactId>
+                <version>${revision}</version>
+            </dependency>
             <dependency>
                 <groupId>de.jplag</groupId>
                 <artifactId>jplag</artifactId>
@@ -209,7 +220,13 @@
         <dependency>
             <groupId>org.junit.jupiter</groupId>
             <artifactId>junit-jupiter-engine</artifactId>
-            <version>5.9.0</version>
+            <version>${junit.version}</version>
+            <scope>test</scope>
+        </dependency>
+        <dependency>
+            <groupId>org.junit.jupiter</groupId>
+            <artifactId>junit-jupiter-params</artifactId>
+            <version>${junit.version}</version>
             <scope>test</scope>
         </dependency>
         <dependency>
@@ -464,8 +481,10 @@
                 <module>jplag.frontend.python-3</module>
                 <module>jplag.frontend.rlang</module>
                 <module>jplag.frontend.rust</module>
+                <module>jplag.frontend.scala</module>
                 <module>jplag.frontend.scheme</module>
                 <module>jplag.frontend.text</module>
+                <module>jplag.endToEndTesting</module>
                 <module>jplag</module>
                 <module>jplag.cli</module>
             </modules>
@@ -487,6 +506,7 @@
                 <module>jplag.frontend.python-3</module>
                 <module>jplag.frontend.rlang</module>
                 <module>jplag.frontend.rust</module>
+                <module>jplag.frontend.scala</module>
                 <module>jplag.frontend.scheme</module>
                 <module>jplag.frontend.text</module>
                 <module>jplag</module>