Browse Source

End To End Test Suite extended (#583)

* comparative values expanded

* adding language specific mapper for the json result paths

* changed static identifier to hashCode. Resources loading made dynamic to be able to create multiple resources folders per language.

* implemented save temporary results with jackson json

* saving the temporary json results for comparison implemented

* change test identifier from an int to string as SHA256 hash

* mapper implemented for the paths

* implementation of storage testing

* insert comments and documentation for functions

* Add comments

* Code cleaned up and comments written. New mapper class implemented for language to path translation

* removed unused code and functions

* ReadMe file extended by "how to add new languages".

* fixed README file

* Changing the json format without DefaultPrettyPrinter() to exclude line ending errors

* fixed formatting

* changed identifier to tested plagiatism file name

* removed sonar cloud RELIABILITY Bugs

* removed code smells detected from sonarcloud

* changed mapper class from HashMap to EnumMap

* removed unused code and added commants for better maintainability

* implemented DynamicTests for OverAllTestCase

* Rework started over complete endToEndTestSuite

* added temporary  save json and read json to module

* removed "result_" from json object

* compare with results implemented.

* cleaned code

* created comments for new classes and functions

* removed code smells and Bugs from code detected by sonarcloud

* changed code format

* Removal of unused code and revision of the naming convention

* Changed testRun to add compaire errors to the list instead of assertEquelas()

* changed code formatting

* removed record constructors and fixed jackson deserialization

* added comments for public and private classes

* removed unnecessary code

* removed code-smells and changed dokumentation in README.md file for new endToEnd suite workflow

* added comment

* removed code-smell

* Made name changes and removed code-smells

* changed return type for helper function

* Update jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/helper/TestSuiteHelper.java

Co-authored-by: Robin Maisch <robinmaisch@users.noreply.github.com>

* Update jplag.endToEndTesting/src/test/java/de/jplag/end_to_end_testing/EndToEndSuiteTest.java

Co-authored-by: Robin Maisch <robinmaisch@users.noreply.github.com>

* Update jplag.endToEndTesting/src/test/java/de/jplag/end_to_end_testing/EndToEndSuiteTest.java

Co-authored-by: Robin Maisch <robinmaisch@users.noreply.github.com>

* Update jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/helper/FileHelper.java

Co-authored-by: Robin Maisch <robinmaisch@users.noreply.github.com>

* Update jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/helper/FileHelper.java

Co-authored-by: Robin Maisch <robinmaisch@users.noreply.github.com>

* Change Proposed Adjustments. Improvement of the code and readability

* Modified language options class included in the endToEnd testing suite

* Changed README file

* fixed result path

* removed code smells detected by sonarcloud

* Functions improved and comments adjusted

Co-authored-by: Robin Maisch <robinmaisch@users.noreply.github.com>
Patrick Thiel 4 years ago
parent
commit
fa071062c9
35 changed files with 2435 additions and 478 deletions
  1. 113 44
      jplag.endToEndTesting/README.md
  2. 12 9
      jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/constants/TestDirectoryConstants.java
  3. 143 0
      jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/helper/FileHelper.java
  4. 0 119
      jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/helper/JPlagTestSuiteHelper.java
  5. 44 11
      jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/helper/JsonHelper.java
  6. 86 0
      jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/helper/TestSuiteHelper.java
  7. 12 0
      jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/model/ExpectedResult.java
  8. 0 52
      jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/model/JsonModel.java
  9. 10 0
      jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/model/Options.java
  10. 34 0
      jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/model/ResultDescription.java
  11. 0 48
      jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/model/ResultModel.java
  12. 0 45
      jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/model/TestCaseModel.java
  13. 258 0
      jplag.endToEndTesting/src/test/java/de/jplag/end_to_end_testing/EndToEndSuiteTest.java
  14. 0 114
      jplag.endToEndTesting/src/test/java/de/jplag/end_to_end_testing/JavaEndToEndTest.java
  15. 0 0
      jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo.java
  16. 0 0
      jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo1.java
  17. 0 0
      jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo1_2.java
  18. 0 0
      jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo1_3.java
  19. 0 0
      jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo1_4.java
  20. 0 0
      jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo1_5.java
  21. 0 0
      jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo1_6.java
  22. 0 0
      jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo2.java
  23. 0 0
      jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo2_5.java
  24. 0 0
      jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo3.java
  25. 0 0
      jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo3_5.java
  26. 0 0
      jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo3_6.java
  27. 0 0
      jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo4.java
  28. 0 0
      jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo4d1.java
  29. 0 0
      jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo4d2.java
  30. 0 0
      jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo4d3.java
  31. 0 0
      jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo5.java
  32. 0 0
      jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo6.java
  33. 0 0
      jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo7.java
  34. 0 36
      jplag.endToEndTesting/src/test/resources/results/JavaResult.json
  35. 1723 0
      jplag.endToEndTesting/src/test/resources/results/java/sortAlgo.json

+ 113 - 44
jplag.endToEndTesting/README.md

@@ -2,18 +2,17 @@
 With the help of the end-to-end module, changes to the detection of JPlag are to be tested.
 With the help of elaborated plagiarisms, which have been worked out from suggestions in the literature on the topic of "plagiarism detection and avoidance", a wide range of detectable change can be covered. The selected plagiarisms are the decisive factor here as to whether a change in recognition can be perceived. 
 
-### References
+## References
 These elaborations provide basic ideas on how a modification of the plagiarized source code can look like or be adapted.
 These code adaptations refer to a wide range of changes starting from
 adding/removing comments to architectural changes in the deliverables.
 
 The following elaborations were used to be able to create the plagiarisms with the largest coverage:
-- [Mossad: defeating software plagiarism detection](https://dl.acm.org/doi/abs/10.1145/3428206 "Mossad: defeating software plagiarism detection")
 - [Detecting Source Code Plagiarism on Introductory Programming Course Assignments Using a Bytecode Approach - Oscar Karnalim](https://ieeexplore.ieee.org/abstract/document/7910274 "Detecting Source Code Plagiarism on Introductory Programming Course Assignments Using a Bytecode Approach - Oscar Karnalim")
 - [Detecting Disguised Plagiarism - Hatem A. Mahmoud](https://arxiv.org/abs/1711.02149 "Detecting Disguised Plagiarism - Hatem A. Mahmoud")
 - [Instructor-centric source code plagiarism detection and plagiarism corpus](https://dl.acm.org/doi/abs/10.1145/2325296.2325328 "Instructor-centric source code plagiarism detection and plagiarism corpus")
 
-### Steps Towards Plagiarism
+## Steps Towards Plagiarism
 The following changes were applied to sample tasks to create test cases:
 <ul type="1">
 	<li>Inserting comments or empty lines (normalization level)</li>
@@ -47,59 +46,129 @@ More detailed information about the create as well as about the subject to the i
 
 Software is according to [§ 2 of the copyright law](https://www.gesetze-im-internet.de/urhg/__2.html "§ 2 of the copyright law") a protected work which may not be plagiarized. 
 
-### JPlag - End To End TestSuite Structure
+## JPlag - End To End TestSuite Structure
 The construction of an end to end test is done with the help of the JPlag api. 
+The tests are generated dynamically according to the existing test data and allow the creation of endToEnd tests for all supported languages of JPlag without having to make any changes to the code.
+The helper loads the existing test data from the designated directory and creates dynamic tests for the individual directories. It is therefore possible to create different test classes for the different languages.
+- JPlagTestSuiteHelper:
 
 ``` java 
-[...]
-    /**
-     * This method creates the necessary results as well as models for a test run and summarizes them for a comparison.
-     * @param testClassNames Plagiarized classes names in the resource directorie which are needed for the test
-     * @param testIdentifier name of the testId to load and identify the stored results
-     * @throws IOException is thrown in case of problems with copying the plagiarism classes
-     * @throws ExitException in case the plagiarism detection with JPlag is preemptively terminated would be of the test.
-     */
-    private void runJPlagTestSuite(String[] testClassNames, int testIdentifier) throws IOException, ExitException {
-        String functionName = StackWalker.getInstance().walk(stream -> stream.skip(1).findFirst().get()).getMethodName();
-        TestCaseModel testCaseModel = jplagTestSuiteHelper.createNewTestCase(testClassNames, functionName);
-        JPlagResult jplagResult = new JPlag(testCaseModel.getJPlagOptionsFromCurrentModel()).run();
-
-        for (JPlagComparison jPlagComparison : jplagResult.getAllComparisons()) {
-            assertEquals(testCaseModel.getCurrentJsonModel().getResultModelById(testIdentifier).getResultSimilarity(), jPlagComparison.similarity(),
-                    "The JPlag results [similarity] do not match the stored values!");
-        }
-    }
-[...]
+public static Map<LanguageOption, Map<String, Path>> getAllLanguageResources()
+```
+
+The list of languages created in this way and their associated data are dynamically generated into test cases in the Test Suite.
+
+``` java
+ Collection<DynamicTest> dynamicOverAllTest()
 ```
-The created plagiarisms are copied with the JPlagTestSuiteHelper into temporary directories, which can then be checked with JPlag. 
 
 In order to be able to distinguish in which domain of the recognition changes have occurred, fine granular test cases are used. These are composed of the changes already mentioned above. The plagiarism is compared with the original delivery and thus it is possible to detect and test small sections of the recognition. 
 
-The values compared so far to detect a match are limited to the similarity of the JPlag scan. It is already planned to expand the comparative values in order to have more indications of a change. 
+The comparative values were discussed and tested. The following results of the JPlag scan are used for the comparison:
+1. minimal similarity as `float`
+2. maximum similarity as `float`
+3. matched token numbe as `int`
 
-The Current Discussion is already taking place and can be found at [End to end testing - "comparative values"](https://github.com/jplag/JPlag/issues/548 "End to end testing - \"comparative values\""). 
+The comparative values were disscussed and elaborated in the issue [End to end testing - "comparative values"](https://github.com/jplag/JPlag/issues/548 "End to end testing - \"comparative values\""). 
+
+Additionally it is possible to create several options for the test data. More information about the test options can be found at [JPlag - option variants for the endToEnd tests #590](https://github.com/jplag/JPlag/issues/590 "JPlag - option variants for the endToEnd tests #590"). Currently, various settings are supported by the `minimumTokenMatch`.  This can be extended as desired in the record class `Options`.
 
 The current JPlag scans will be compared with the stored ones.
-This was done by storing the data in a *.json file which is read at the beginning of each test run. 
+This was done by storing the data in a *.json file which is read at the beginning of each test run.
 
 ``` json  
-{
 [...]
-    "function_name": "normalizationLevelTest",
-    "test_results": [
-      {
-        "result_similarity": 100,
-        "test_identifier": 0
-      },
-      {
-        "result_similarity": 100,
-        "test_identifier": 1
-      },
-      {
-        "result_similarity": 100,
-        "test_identifier": 2
-      }
-    ]
-  }
+{
+  "options" : {
+    "minimum_token_match" : 1
+  },
+  "tests" : {
+    "SortAlgo-SortAlgo5" : {
+      "minimal_similarity" : 82.14286,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo-SortAlgo6" : {
+      "minimal_similarity" : 83.58209,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo-SortAlgo7" : {
+      "minimal_similarity" : 96.42857,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 54
+    },
 [...]
 ```
+--- 
+
+## Create New Language End To End Tests
+
+This section explains how to create new end to end tests in the existing test suite. 
+### Creating The Plagiarism
+Before you add a new language to the end to end tests i would like to point out that the quality of the tests depends dreadfully on the plagiarism techniques you choose wicht were explaint in sechtion [Steps Towards Plagiarism](#steps-towards-plagiarism).
+If you need more information about the creation of plans for this purpose, you can also read the elaborations that can be found under [References](#references).
+The more and varied changes you apply, the more accurate the end-to-end tests for the language will be.
+
+In the following an example is shown which is in the JavaEndToEnd tests and is used.
+
+**Changing control structures for(…) to while(…):**
+
+``` java
+//base class
+public class SortAlgo {
+//[...]
+public void BubbleSortWithoutRecursion(Integer arr[]) {
+		for(int i = arr.length; i > 1 ; i--) {
+			for(int innerCounter = 0; innerCounter < arr.length-1; innerCounter++)
+			{
+				if (arr[innerCounter] > arr[innerCounter + 1]) {
+					swap(arr, innerCounter, (innerCounter + 1));
+				}
+			}
+		}
+	}
+//[...]
+}
+```
+
+``` java
+//created plagiarism
+public class SortAlgo5{
+//[...]
+public void BubbleSortWithoutRecursion(Integer arr[]) {
+		int i = arr.length;
+		while(i > 1)
+		{
+			int innerCounter = 0;
+			while(innerCounter < arr.length -1)
+			{
+				if (arr[innerCounter] > arr[innerCounter + 1]) {
+					swap(arr, innerCounter, (innerCounter + 1));
+				}
+				innerCounter++;
+			}
+			i--;
+		}
+	}
+//[...]
+}
+```
+### Copying Plagiarism To The Resources
+
+The plagiarisms created in [Creating The Plagiarism](#creating-the-plagiarism) must now be copied to the corresponding resources folder. It is important not to mix the languages of the plagiarisms or to copy the data into bottle resource paths.
+
+- At the path `JPlag\jplag.endToEndTesting\src\test\resources\languageTestFiles` a new folder for the language should be created if it does not already exist. For example `[...]\resources\languageTestFiles\JAVA`. If you have plagiarized several different code samples, you can also create additional subfolders under the newly created folder for example `[...]\resources\languageTestFiles\JAVA\sortAlgo`.
+
+It is important to note that the resource folder name must be exactly the same as the language identifier name in JPlag/Language. Otherwise the language option cannot be parsed correctly to the enum-type.
+ - c++ with "cpp"
+ - c# with "csharp"
+ - GO with "go"
+ - Java with "java"
+ - Kotlin with "kotlin"
+ - Python3 with "python3"
+ - R with "rlang"
+ - Rust with "rust"
+ - Scala with "scala"
+
+Once the tests have been run for the first time, the information for the tests is stored in the folder `..\target\testing-directory-submission\LANGUAGE`.  This data can be copied to the path `[...]\resources\results\LANGUAGE`. Each subdirectory gets its own result json file as `[...]\resources\results\JAVA\sortAlgo.json`. Once the test data has been copied, the endToEnd tests can be successfully tested. As soon as a change in the detection takes place, the results will differ from the stored results and the tests will fail if the results have changed.

+ 12 - 9
jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/constants/TestDirectoryConstants.java

@@ -12,21 +12,24 @@ public final class TestDirectoryConstants {
     }
 
     /**
-     * Name for the folder to copy the submissions to in order to test them with JPlag
+     * Create the complete path to the temporary result files. Here the temporary system path is extended with the
+     * "RESULT_DIRECTORY_NAME", which is predefined in this class.
      */
-    public static final String TEMPORARY_DIRECTORY_NAME = "testing-directory-submission";
+    public static final Path TEMPORARY_RESULT_DIRECTORY_NAME = Path.of("target", "testing-directory-temporary-result");
+
     /**
-     * Create the complete path to the submission files. Here the temporary system path is extended with the
-     * "TEMPORARY_DIRECTORY_NAME", which is predefined in this class.
+     * Base path to the saved results
      */
-    public static final String TEMPORARY_SUBMISSION_DIRECTORY_NAME = Path.of("target", TEMPORARY_DIRECTORY_NAME).toString();
+    public static final Path BASE_PATH_TO_RESULT_JSON = Path.of("src", "test", "resources", "results");
 
     /**
-     * Base path to the created plagiarism and the main file located in the project resources.
+     * Base path to the endToEnd testing resources
      */
-    public static final Path BASE_PATH_TO_JAVA_RESOURCES_SORTALGO = Path.of("src", "test", "resources", "java", "sortAlgo");
+    public static final Path BASE_PATH_TO_LANGUAGE_RESOURCES = Path.of("src", "test", "resources", "languageTestFiles");
+
     /**
-     * Base path to the saved results of the previous tests in a *.json file
+     * Create the complete path to the submission files. Here the temporary system path is extended with the
+     * "SUBMISSION_DIRECTORY_NAME", which is predefined in this class.
      */
-    public static final Path BASE_PATH_TO_JAVA_RESULT_JSON = Path.of("src", "test", "resources", "results", "JavaResult.json");
+    public static final Path TEMPORARY_SUBMISSION_DIRECTORY_NAME = Path.of("target", "testing-directory-submission");
 }

+ 143 - 0
jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/helper/FileHelper.java

@@ -0,0 +1,143 @@
+package de.jplag.end_to_end_testing.helper;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.StandardCopyOption;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.List;
+import java.util.Objects;
+import java.util.stream.Collectors;
+
+import de.jplag.end_to_end_testing.constants.TestDirectoryConstants;
+
+/**
+ * Helper class to perform all necessary operations or functions on files or folders.
+ */
+public class FileHelper {
+
+    private FileHelper() {
+        // private constructor to prevent instantiation
+    }
+
+    /**
+     * Merges all contained filenames together without extension
+     * @param files whose names are to be merged
+     * @return merged filenames
+     */
+    public static String getEnclosedFileNamesFromCollection(Collection<File> files) {
+
+        return files.stream().map(File::getName).map(fileName -> fileName.substring(0, fileName.lastIndexOf('.'))).collect(Collectors.joining());
+    }
+
+    /**
+     * Load all possible languages in resource path
+     * @param directoryNames folder names for which the language options should be listed.
+     * @return list of all LanguageOptions included in the resource path
+     */
+    public static List<String> getLanguageOptionsFromPath(String[] directoryNames) {
+        return Arrays.stream(directoryNames).map(language -> language).filter(Objects::nonNull).toList();
+    }
+
+    /**
+     * @param directorieRoot path from which all folders should be loaded
+     * @return all folders found in the specified path
+     */
+    public static String[] getAllDirectoriesInPath(Path directorieRoot) {
+        return directorieRoot.toFile().list((dir, name) -> new File(dir, name).isDirectory());
+    }
+
+    /**
+     * Copies the passed filenames to a temporary path to use them in the tests
+     * @param classNames for which the test case is to be created
+     * @return paths created to the test submissions
+     * @throws IOException Exception can be thrown in cases that involve reading, copying or locating files.
+     */
+    public static String[] createNewTestCaseDirectory(String[] classNames) throws IOException {
+        // Copy the resources data to the temporary path
+        String[] returnSubmissionPath = new String[classNames.length];
+        for (int counter = 0; counter < classNames.length; counter++) {
+            Path originalPath = Path.of(classNames[counter]);
+            returnSubmissionPath[counter] = Path
+                    .of(TestDirectoryConstants.TEMPORARY_SUBMISSION_DIRECTORY_NAME.toString(), "submission" + (counter + 1)).toAbsolutePath()
+                    .toString();
+            Path copyPath = Path.of(TestDirectoryConstants.TEMPORARY_SUBMISSION_DIRECTORY_NAME.toString(), "submission" + (counter + 1),
+                    originalPath.getFileName().toString());
+
+            File directory = new File(copyPath.toString());
+            if (!directory.exists()) {
+                directory.mkdirs();
+            }
+            Files.copy(originalPath, copyPath, StandardCopyOption.REPLACE_EXISTING);
+        }
+        return returnSubmissionPath;
+    }
+
+    /**
+     * Delete directory with including files
+     * @param folder Path to a folder or file to be deleted. This happens recursively to the path
+     * @throws IOException if an I/O error occurs
+     */
+    public static void deleteCopiedFiles(File folder) throws IOException {
+        if (!folder.exists()) {
+            return;
+        }
+        File[] files = folder.listFiles();
+        if (files == null) { // some JVMs return null for empty dirs
+            return;
+        }
+        for (File file : files) {
+            if (file.isDirectory()) {
+                deleteCopiedFiles(file);
+            } else {
+                Files.delete(file.toPath());
+            }
+        }
+        Files.delete(folder.toPath());
+    }
+
+    /**
+     * Creates directory if it dose not exist
+     * @param directory to be created
+     * @throws IOException if the directory could not be created
+     */
+    public static void createDirectoryIfItDoesNotExist(File directory) throws IOException {
+        if (!directory.exists() && !directory.mkdirs()) {
+            throw new IOException(createNewIOExceptionStringForFileOrFOlderCreation(directory));
+        }
+    }
+
+    /**
+     * Creates file if it dose not exist
+     * @param file to be created
+     * @throws IOException if the file could not be created
+     */
+    public static void createFileIfItDoesNotExist(File file) throws IOException {
+        if (!file.exists() && !file.createNewFile()) {
+            throw new IOException(createNewIOExceptionStringForFileOrFOlderCreation(file));
+        }
+    }
+
+    /**
+     * @param resourcenPaths list of paths that lead to test resources
+     * @return all filenames contained in the paths
+     */
+    public static String[] loadAllTestFileNames(Path resourcenPaths) {
+        var files = resourcenPaths.toFile().listFiles();
+        String[] fileNames = new String[files.length];
+        for (int i = 0; i < files.length; i++) {
+            fileNames[i] = files[i].getName();
+        }
+        return fileNames;
+    }
+
+    /**
+     * @param file for which the exception text is to be created
+     * @return exception text for the specified file
+     */
+    private static String createNewIOExceptionStringForFileOrFOlderCreation(File file) {
+        return "The file/folder at the location [" + file.toString() + "] could not be created!";
+    }
+}

+ 0 - 119
jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/helper/JPlagTestSuiteHelper.java

@@ -1,119 +0,0 @@
-package de.jplag.end_to_end_testing.helper;
-
-import java.io.File;
-import java.io.FileNotFoundException;
-import java.io.IOException;
-import java.nio.file.Files;
-import java.nio.file.Path;
-import java.nio.file.StandardCopyOption;
-import java.util.Arrays;
-import java.util.List;
-
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import de.jplag.end_to_end_testing.constants.TestDirectoryConstants;
-import de.jplag.end_to_end_testing.model.JsonModel;
-import de.jplag.end_to_end_testing.model.TestCaseModel;
-
-/**
- * This helper class deals with creating the test cases as well as copying and deleting for the end-to-end tests. The
- * required plagiarisms are copied from the resource folder to a temporary location, thus creating a folder structure
- * that can be tested by JPlag. Models are instantiated here and required information for the tests is loaded.
- */
-public class JPlagTestSuiteHelper {
-
-    private static final Logger logger = LoggerFactory.getLogger(JPlagTestSuiteHelper.class);
-
-    private String[] resourceNames;
-    private List<JsonModel> resultModel;
-    private String languageIdentifier;
-
-    /**
-     * Helper class for the endToEnd tests. In this class the necessary resources are loaded, prepared and copied for the
-     * tests based on the passed parameters. An instance of this class loads all necessary paths and properties for a test
-     * run with the specified language
-     * @param languageIdentifier for loading language-specific resources
-     * @throws IOException is thrown for all problems that may occur while parsing the json file. This includes both reading
-     * and parsing problems.
-     */
-    public JPlagTestSuiteHelper(String languageIdentifier) throws IOException {
-        this.languageIdentifier = languageIdentifier;
-        this.resourceNames = new File(TestDirectoryConstants.BASE_PATH_TO_JAVA_RESOURCES_SORTALGO.toString()).list();
-
-        this.resultModel = JsonHelper.getResultModelFromPath();
-        logger.debug("temp path at [{}]", TestDirectoryConstants.TEMPORARY_SUBMISSION_DIRECTORY_NAME);
-    }
-
-    /**
-     * creates all necessary folder paths and objects for a test run. Also searches for the stored previous results of the
-     * test in order to compare them with the current results.
-     * @param classNames Array of class names with language specific extension to be prepared for a test.
-     * @return comparison results saved for the test
-     * @throws IOException Exception can be thrown in cases that involve reading, copying or locating files.
-     */
-    public TestCaseModel createNewTestCase(String[] classNames, String functionName) throws IOException {
-        createNewTestCaseDirectory(classNames);
-        JsonModel resultJsonModel = resultModel.stream().filter(jsonModel -> functionName.equals(jsonModel.getFunctionName())).findAny().orElse(null);
-        return new TestCaseModel(TestDirectoryConstants.TEMPORARY_SUBMISSION_DIRECTORY_NAME, resultJsonModel, languageIdentifier);
-    }
-
-    /**
-     * The copied data should be deleted after instance closure
-     * @throws IOException if an I/O error occurs
-     */
-    public void clear() throws IOException {
-        logger.debug("Class instance was cleaned!");
-        deleteCopiedFiles(new File(TestDirectoryConstants.TEMPORARY_SUBMISSION_DIRECTORY_NAME));
-    }
-
-    /**
-     * Copies the passed filenames to a temporary path to use them in the tests
-     * @throws IOException Exception can be thrown in cases that involve reading, copying or locating files.
-     */
-    private void createNewTestCaseDirectory(String[] classNames) throws IOException {
-        // before copying files to the test path, check if all files are in the resource
-        // directory
-        for (String className : classNames) {
-            if (!Arrays.asList(resourceNames).contains(className)) {
-                throw new FileNotFoundException(String.format("The specified class could not be found! [%s]", className));
-            }
-        }
-        // Copy the resources data to the temporary path
-        for (int counter = 0; counter < classNames.length; counter++) {
-            Path originalPath = Path.of(TestDirectoryConstants.BASE_PATH_TO_JAVA_RESOURCES_SORTALGO.toString(), classNames[counter]);
-            Path copiePath = Path.of(TestDirectoryConstants.TEMPORARY_SUBMISSION_DIRECTORY_NAME,
-                    TestDirectoryConstants.TEMPORARY_DIRECTORY_NAME + (counter + 1), classNames[counter]);
-
-            File directory = new File(copiePath.toString());
-            if (!directory.exists()) {
-                directory.mkdirs();
-            }
-
-            Files.copy(originalPath, copiePath, StandardCopyOption.REPLACE_EXISTING);
-            logger.debug("Copy file from [{}] to [{}]", originalPath, copiePath);
-        }
-    }
-
-    /**
-     * Delete directory with including files
-     * @param folder Path to a folder or file to be deleted. This happens recursively to the path
-     * @throws IOException if an I/O error occurs
-     */
-    private void deleteCopiedFiles(File folder) throws IOException {
-        File[] files = folder.listFiles();
-        if (files != null) { // some JVMs return null for empty dirs
-            for (File file : files) {
-                if (file.isDirectory()) {
-                    deleteCopiedFiles(file);
-                } else {
-                    Files.delete(file.toPath());
-                    logger.debug("Delete file in folder: [{}]", file);
-                }
-            }
-        }
-        Files.delete(folder.toPath());
-        logger.debug("Delete folder: [{}]", folder);
-    }
-
-}

+ 44 - 11
jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/helper/JsonHelper.java

@@ -1,19 +1,18 @@
 package de.jplag.end_to_end_testing.helper;
 
 import java.io.IOException;
+import java.nio.file.Path;
 import java.util.Arrays;
+import java.util.Collections;
 import java.util.List;
 
 import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.ObjectWriter;
 
 import de.jplag.end_to_end_testing.constants.TestDirectoryConstants;
-import de.jplag.end_to_end_testing.model.JsonModel;
-
-/**
- * Helper class for serializing and creating all json dependent events.
- */
-public final class JsonHelper {
+import de.jplag.end_to_end_testing.model.ResultDescription;
 
+public class JsonHelper {
     /**
      * private constructor to prevent instantiation
      */
@@ -22,12 +21,46 @@ public final class JsonHelper {
     }
 
     /**
-     * Parsing the old results in the json file as a list from ResultJsonModel.
-     * @return list of saved results for the test cases
+     * @param directoryName name to the result path
+     * @param languageIdentifier for which the results are to be loaded
+     * @return ResultDescription as serialized object
      * @throws IOException is thrown for all problems that may occur while parsing the json file. This includes both reading
-     * and parsing problems.
      */
-    public static List<JsonModel> getResultModelFromPath() throws IOException {
-        return Arrays.asList(new ObjectMapper().readValue(TestDirectoryConstants.BASE_PATH_TO_JAVA_RESULT_JSON.toFile(), JsonModel[].class));
+    public static List<ResultDescription> getJsonModelListFromPath(String directoryName, String languageIdentifier) throws IOException {
+
+        Path jsonPath = Path.of(TestDirectoryConstants.BASE_PATH_TO_RESULT_JSON.toString(), languageIdentifier, directoryName + ".json");
+
+        if (jsonPath.toFile().exists() && jsonPath.toFile().length() > 0) {
+
+            return Arrays.asList(new ObjectMapper().readValue(jsonPath.toFile(), ResultDescription[].class));
+        } else {
+            return Collections.<ResultDescription>emptyList();
+        }
+    }
+
+    /**
+     * Saves the passed object as a json file to the given path
+     * @param resultDescriptionist list of elements to be saved
+     * @param directoryName path to the temporary storage location
+     * @param languageIdentifier for which the results should be stored
+     * @throws IOException Signals that an I/O exception of some sort has occurred. Thisclass is the general class of
+     * exceptions produced by failed orinterrupted I/O operations.
+     */
+    public static void writeJsonModelsToJsonFile(List<ResultDescription> resultDescriptionist, String directoryName, String languageIdentifier)
+            throws IOException {
+        // create an instance of DefaultPrettyPrinter
+        // new DefaultPrettyPrinter()
+        ObjectWriter writer = new ObjectMapper().writer().withDefaultPrettyPrinter();
+
+        Path temporaryDirectory = Path.of(TestDirectoryConstants.TEMPORARY_SUBMISSION_DIRECTORY_NAME.toString(), languageIdentifier,
+                directoryName + ".json");
+
+        FileHelper.createDirectoryIfItDoesNotExist(temporaryDirectory.getParent().toFile());
+        FileHelper.createFileIfItDoesNotExist(temporaryDirectory.toFile());
+
+        // convert book object to JSON file
+
+        writer.writeValue(temporaryDirectory.toFile(), resultDescriptionist.toArray());
+
     }
 }

+ 86 - 0
jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/helper/TestSuiteHelper.java

@@ -0,0 +1,86 @@
+package de.jplag.end_to_end_testing.helper;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.file.Path;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import de.jplag.JPlagComparison;
+import de.jplag.Submission;
+import de.jplag.end_to_end_testing.constants.TestDirectoryConstants;
+
+public class TestSuiteHelper {
+
+    /**
+     * private constructor to prevent instantiation
+     */
+    private TestSuiteHelper() {
+        // For Serialization
+    }
+
+    /**
+     * Loads all existing test data into a test structure. the complex structure consists of the LanguageOption and their
+     * data to be tested. These are divided into folders. for more information please read the README file
+     * @return mapped LanguageOption to the data under test
+     */
+    public static Map<String, Map<String, Path>> getAllLanguageResources() {
+        String[] languageDirectoryNames = FileHelper.getAllDirectoriesInPath(TestDirectoryConstants.BASE_PATH_TO_LANGUAGE_RESOURCES);
+        List<String> languageInPathList = FileHelper.getLanguageOptionsFromPath(languageDirectoryNames);
+
+        Map<String, Map<String, Path>> returnMap = new HashMap<>();
+
+        for (String languageIdentifier : languageInPathList) {
+            var tempMap = new HashMap<String, Path>();
+            var allDirectoriesInPath = FileHelper
+                    .getAllDirectoriesInPath(Path.of(TestDirectoryConstants.BASE_PATH_TO_LANGUAGE_RESOURCES.toString(), languageIdentifier));
+            Arrays.asList(allDirectoriesInPath).forEach(directory -> tempMap.put(Path.of(directory).toFile().getName(),
+                    Path.of(TestDirectoryConstants.BASE_PATH_TO_LANGUAGE_RESOURCES.toString(), languageIdentifier, directory)));
+            returnMap.put(languageIdentifier, tempMap);
+        }
+        return returnMap;
+    }
+
+    /**
+     * Creates a unique identifier from the submissions in the JPlagComparison object which is used to find the results in
+     * the json files.
+     * @param jPlagComparison object from which the hash should be generated
+     * @return unique identifier for test case recognition
+     */
+    public static String getTestIdentifier(JPlagComparison jPlagComparison) {
+
+        return List.of(jPlagComparison.getFirstSubmission(), jPlagComparison.getSecondSubmission()).stream().map(Submission::getFiles)
+                .map(FileHelper::getEnclosedFileNamesFromCollection).sorted().collect(Collectors.joining("-"));
+
+    }
+
+    /**
+     * Creates the permutation of all data contained in the passed parameters and adds it to the given path.
+     * @param fileNames for which the permutations are needed
+     * @param path to which the permutations are to be copied
+     * @return all permutations of the specified files to the path specified
+     */
+    public static List<String[]> getTestCases(String[] fileNames, Path path) {
+        ArrayList<String[]> testCases = new ArrayList<>();
+
+        for (int outerCounter = 0; outerCounter < fileNames.length; outerCounter++) {
+            for (int innerCounter = outerCounter + 1; innerCounter < fileNames.length; innerCounter++) {
+                testCases.add(new String[] {Path.of(path.toAbsolutePath().toString(), fileNames[outerCounter]).toString(),
+                        Path.of(path.toAbsolutePath().toString(), fileNames[innerCounter]).toString()});
+            }
+        }
+        return testCases;
+    }
+
+    /**
+     * The copied data should be deleted after instance closure
+     * @throws IOException if an I/O error occurs
+     */
+    public static void clear() throws IOException {
+        FileHelper.deleteCopiedFiles(new File(TestDirectoryConstants.TEMPORARY_SUBMISSION_DIRECTORY_NAME.toString()));
+    }
+}

+ 12 - 0
jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/model/ExpectedResult.java

@@ -0,0 +1,12 @@
+package de.jplag.end_to_end_testing.model;
+
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+/**
+ * contains the current comparative values for the endToEnd tests. The comparative values were determined by discussion
+ * which can be found at https://github.com/jplag/JPlag/issues/548 Here this object is used for serialization and
+ * deserialization of the information from json to object or object to json.
+ */
+public record ExpectedResult(@JsonProperty("minimal_similarity") float resultSimilarityMinimum,
+        @JsonProperty("maximum_similarity") float resultSimilarityMaximum, @JsonProperty("matched_token_number") int resultMatchedTokenNumber) {
+}

+ 0 - 52
jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/model/JsonModel.java

@@ -1,52 +0,0 @@
-package de.jplag.end_to_end_testing.model;
-
-import java.util.Arrays;
-
-import com.fasterxml.jackson.annotation.JsonProperty;
-
-/**
- * The ResultJsonModel is the java object for the JavaResult.json file. The object contains all the necessary
- * information for the comparisons in the test cases between old and new results, which have been stored in the
- * JavaResult.json file.
- */
-public class JsonModel {
-    @JsonProperty("function_name")
-    private String functionName;
-    @JsonProperty("test_results")
-    private ResultModel[] results;
-
-    /**
-     * Constructor for the JsonModel. The model is the serialization of the Json file in the form of a Java object.
-     * @param functionName the function name for the associated test results. Used as identifier to search results for the
-     * test cases.
-     * @param results Collection of the results that are in the current json file for certain tests
-     */
-    public JsonModel(String functionName, ResultModel[] results) {
-        this.functionName = functionName;
-        this.results = results;
-    }
-
-    /**
-     * empty constructor in case the serialization contains an empty object to prevent throwing exceptions. this constructor
-     * was necessary for serialization with the Jackson parse extension
-     */
-    public JsonModel() {
-        // For Serialization
-    }
-
-    /**
-     * @return the name of the currently used function stored in json
-     */
-    public String getFunctionName() {
-        return functionName;
-    }
-
-    /**
-     * returns the results for the function that have been stored for the given id.
-     * @param identifier for the comparative values
-     * @return associated comparison values that have been assigned to the identifier
-     */
-    public ResultModel getResultModelById(Integer identifier) {
-        return Arrays.asList(results).stream().filter(resultModel -> identifier.equals(resultModel.getTestId())).findAny().orElse(null);
-    }
-}

+ 10 - 0
jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/model/Options.java

@@ -0,0 +1,10 @@
+package de.jplag.end_to_end_testing.model;
+
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+/**
+ * The object contains required options for the endToEndt tests which are important for the test suite. The options were
+ * determined by discussion which can be found at https://github.com/jplag/JPlag/issues/590.
+ */
+public record Options(@JsonProperty("minimum_token_match") Integer minimumTokenMatch) {
+}

+ 34 - 0
jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/model/ResultDescription.java

@@ -0,0 +1,34 @@
+package de.jplag.end_to_end_testing.model;
+
+import java.util.Map;
+
+import com.fasterxml.jackson.annotation.JsonIgnore;
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+/**
+ * Object that maps the results of the end top end tests using the identifierToResultMap. this creates a map of test
+ * data and its results for each possible option specified. this is important both for serializing the data into json
+ * format and for deserialization.
+ */
+public record ResultDescription(@JsonIgnore String languageIdentifier, @JsonProperty("options") Options options,
+        @JsonProperty("tests") Map<String, ExpectedResult> identifierToResultMap) {
+
+    /**
+     * @param identifier for which the stored results are needed
+     * @return stored results as ExpectedResult object for the passed id
+     */
+    @JsonIgnore
+    public ExpectedResult getExpectedResultByIdentifier(String identifier) {
+        return identifierToResultMap.get(identifier);
+    }
+
+    /**
+     * Adds expected results to the existing list of the object. These results are stored in a map with the given
+     * identifier.
+     * @param identifier under which the results should be stored
+     * @param expectedResult expected results belonging to the identifier
+     */
+    public void putIdentifierToResultMap(String identifier, ExpectedResult expectedResult) {
+        identifierToResultMap.put(identifier, expectedResult);
+    }
+}

+ 0 - 48
jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/model/ResultModel.java

@@ -1,48 +0,0 @@
-package de.jplag.end_to_end_testing.model;
-
-import com.fasterxml.jackson.annotation.JsonProperty;
-
-/**
- * The ResultModel contains all information and comparison values that are important for a test and that could be parsed
- * from the json file.
- */
-public class ResultModel {
-
-    @JsonProperty("result_similarity")
-    private float resultSimilarity;
-    @JsonProperty("test_identifier")
-    private int testIdentifier;
-
-    /**
-     * Constructor for the ResultModel. The model is the serialization of the Json file in the form of a Java object.
-     * @param resultSimilarity stored comparative value of the similarity
-     * @param testIdentifier specifies which associated test results are needed for a test case and are therefore associated
-     * with this Id.
-     */
-    public ResultModel(float resultSimilarity, int testIdentifier) {
-        this.resultSimilarity = resultSimilarity;
-        this.testIdentifier = testIdentifier;
-    }
-
-    /**
-     * empty constructor in case the serialization contains an empty object to prevent throwing exceptions. this constructor
-     * was necessary for serialization with the Jackson parse extension
-     */
-    public ResultModel() {
-        // For Serialization
-    }
-
-    /**
-     * @return Identifier assigned to the result
-     */
-    public int getTestId() {
-        return testIdentifier;
-    }
-
-    /**
-     * @return of the comparative similarity
-     */
-    public float getResultSimilarity() {
-        return resultSimilarity;
-    }
-}

+ 0 - 45
jplag.endToEndTesting/src/main/java/de/jplag/end_to_end_testing/model/TestCaseModel.java

@@ -1,45 +0,0 @@
-package de.jplag.end_to_end_testing.model;
-
-import java.util.ArrayList;
-import java.util.Arrays;
-
-import de.jplag.options.JPlagOptions;
-
-/**
- * Median class for trading the results and the paths used in the tests. Also the JPlagOptions used for a test run are
- * created here.
- */
-public class TestCaseModel {
-
-    private JsonModel jsonModel;
-    private String languageIdentifier;
-    private String submissionFolderPath;
-
-    /**
-     * Constructor for creating the instance
-     * @param submissionFolderPath path to the plagiarism folder structure to be tested using Jplag
-     * @param jsonModel the parsed result model needed for the test to be able to match it later in the tests.
-     * @param languageIdentifier the language selection in which the tests take place
-     */
-    public TestCaseModel(String submissionFolderPath, JsonModel jsonModel, String languageIdentifier) {
-        this.submissionFolderPath = submissionFolderPath;
-        this.jsonModel = jsonModel;
-        this.languageIdentifier = languageIdentifier;
-    }
-
-    /**
-     * returns the, for the test, current result model that has been persisted in the saved list.
-     * @return ResultJsonModel loaded for the test
-     */
-    public JsonModel getCurrentJsonModel() {
-        return jsonModel;
-    }
-
-    /**
-     * creates the JPlag options for the test run from the values passed in the constructor.
-     * @return current JPlag options for the created object
-     */
-    public JPlagOptions getJPlagOptionsFromCurrentModel() {
-        return new JPlagOptions(new ArrayList<>(Arrays.asList(submissionFolderPath)), new ArrayList<>(), languageIdentifier);
-    }
-}

+ 258 - 0
jplag.endToEndTesting/src/test/java/de/jplag/end_to_end_testing/EndToEndSuiteTest.java

@@ -0,0 +1,258 @@
+package de.jplag.end_to_end_testing;
+
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.io.IOException;
+import java.nio.file.Path;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Optional;
+import java.util.StringJoiner;
+
+import org.junit.jupiter.api.AfterAll;
+import org.junit.jupiter.api.DynamicTest;
+import org.junit.jupiter.api.TestFactory;
+
+import de.jplag.JPlag;
+import de.jplag.JPlagComparison;
+import de.jplag.JPlagResult;
+import de.jplag.end_to_end_testing.helper.FileHelper;
+import de.jplag.end_to_end_testing.helper.JsonHelper;
+import de.jplag.end_to_end_testing.helper.TestSuiteHelper;
+import de.jplag.end_to_end_testing.model.*;
+import de.jplag.exceptions.ExitException;
+import de.jplag.options.JPlagOptions;
+
+/**
+ * Main test class for end-to-end testing in all language. The test cases aim to detect changes in the detection of
+ * plagiarism in the Java language and to be able to roughly categorize them. The plagiarism is compared with the
+ * original class. The results are compared with the results from previous tests and changes are detected.
+ */
+public class EndToEndSuiteTest {
+    // Language -> directory names and Paths
+    private Map<String, Map<String, Path>> LanguageToTestCaseMapper;
+
+    private List<Options> options;
+
+    private static Map<String, List<ResultDescription>> temporaryResultList;
+    private static List<String> validationErrors;
+    private static String languageIdentifier;
+
+    public EndToEndSuiteTest() throws IOException {
+        // Loading the test resources
+        LanguageToTestCaseMapper = TestSuiteHelper.getAllLanguageResources();
+        // creating the temporary lists for the test run
+        validationErrors = new ArrayList<>();
+        temporaryResultList = new HashMap<>();
+        // creating options object for the testSuite
+        setRunOptions();
+    }
+
+    /**
+     * creating the required options object for the endToEnd tests
+     */
+    private void setRunOptions() {
+        options = new ArrayList<>();
+        options.add(new Options(1));
+        options.add(new Options(15));
+    }
+
+    /**
+     * Creates the result json files based on the current test results after the test run.
+     * @throws IOException is thrown for all problems that may occur while parsing the json file. This includes both reading
+     */
+    @AfterAll
+    public static void tearDown() throws IOException {
+        for (var resultDescriptionItem : temporaryResultList.entrySet()) {
+            JsonHelper.writeJsonModelsToJsonFile(resultDescriptionItem.getValue(), resultDescriptionItem.getKey(), languageIdentifier);
+        }
+    }
+
+    /**
+     * Creates the test cases over all language options for which data is available and the current test options.
+     * @return dynamic test cases across all test data and languages
+     * @throws IOException is thrown for all problems that may occur while parsing the json file. This includes both reading
+     */
+    @TestFactory
+    Collection<DynamicTest> dynamicOverAllTest() throws IOException {
+        for (Entry<String, Map<String, Path>> languageMap : LanguageToTestCaseMapper.entrySet()) {
+            String currentLanguageIdentifier = languageMap.getKey();
+            for (Entry<String, Path> languagePaths : languageMap.getValue().entrySet()) {
+                String[] fileNames = FileHelper.loadAllTestFileNames(languagePaths.getValue());
+                var testCases = TestSuiteHelper.getTestCases(fileNames, languagePaths.getValue());
+                var testCollection = new ArrayList<DynamicTest>();
+                String directoryName = languagePaths.getValue().getFileName().toString();
+                List<ResultDescription> tempResult = JsonHelper.getJsonModelListFromPath(directoryName, currentLanguageIdentifier);
+                languageIdentifier = currentLanguageIdentifier;
+                for (Options option : options) {
+                    for (var testCase : testCases) {
+                        Optional<ResultDescription> currentResultDescription = tempResult.stream().filter(x -> x.options().equals(option))
+                                .findFirst();
+                        testCollection.add(DynamicTest.dynamicTest(getTestCaseDisplayName(option, languageIdentifier, testCase), () -> {
+                            runTests(directoryName, option, currentLanguageIdentifier, testCase, currentResultDescription);
+                        }));
+                    }
+                }
+                return testCollection;
+            }
+        }
+        return null;
+    }
+
+    /**
+     * Superordinate test function to be able to continue to check all data to be tested in case of failed tests
+     * @param directoryName name of the current tested directory
+     * @param options for the current test run
+     * @param currentLanguageIdentifier current JPlag language option
+     * @param testFiles files to be tested
+     * @param currentResultDescription results stored for the test data
+     * @throws IOException Signals that an I/O exception of some sort has occurred. Thisclass is the general class of
+     * exceptions produced by failed orinterrupted I/O operations
+     * @throws ExitException Exceptions for problems during the execution of JPlag that lead to an preemptive exit.
+     */
+    private void runTests(String directoryName, Options option, String currentLanguageIdentifier, String[] testFiles,
+            Optional<ResultDescription> currentResultDescription) throws IOException, ExitException {
+        try {
+            ResultDescription currentResult = currentResultDescription.orElse(null);
+            runJPlagTestSuite(directoryName, option, currentLanguageIdentifier, testFiles, currentResult);
+        } finally {
+            validationErrors.clear();
+            TestSuiteHelper.clear();
+        }
+    }
+
+    /**
+     * EndToEnd test for the passed objects
+     * @param directoryName name of the current tested directory
+     * @param options for the current test run
+     * @param languageIdentifier current JPlag language option
+     * @param testFiles files to be tested
+     * @param currentResultDescription results stored for the test data
+     * @throws IOException Signals that an I/O exception of some sort has occurred. Thisclass is the general class of
+     * exceptions produced by failed orinterrupted I/O operations
+     * @throws ExitException Exceptions for problems during the execution of JPlag that lead to an preemptive exit.
+     */
+    private void runJPlagTestSuite(String directoryName, Options options, String languageIdentifier, String[] testFiles,
+            ResultDescription currentResultDescription) throws IOException, ExitException {
+        String[] submissionPath = FileHelper.createNewTestCaseDirectory(testFiles);
+
+        JPlagOptions jplagOptions = new JPlagOptions(Arrays.asList(submissionPath), new ArrayList<>(), languageIdentifier);
+
+        jplagOptions.setMinimumTokenMatch(options.minimumTokenMatch());
+
+        JPlagResult jplagResult = new JPlag(jplagOptions).run();
+
+        List<JPlagComparison> currentJPlagComparison = jplagResult.getAllComparisons();
+
+        for (JPlagComparison jPlagComparison : currentJPlagComparison) {
+            String identifier = TestSuiteHelper.getTestIdentifier(jPlagComparison);
+            addToTemporaryResultMap(directoryName, options, jPlagComparison, languageIdentifier);
+
+            assertNotNull(currentResultDescription, "No stored result could be found for the current LanguageOption! " + options.toString());
+
+            ExpectedResult result = currentResultDescription.getExpectedResultByIdentifier(TestSuiteHelper.getTestIdentifier(jPlagComparison));
+            assertNotNull(result, "No stored result could be found for the identifier! " + identifier);
+
+            if (Float.compare(result.resultSimilarityMinimum(), jPlagComparison.minimalSimilarity()) != 0) {
+                addToValidationErrors("minimalSimilarity", String.valueOf(result.resultSimilarityMinimum()),
+                        String.valueOf(jPlagComparison.minimalSimilarity()));
+            }
+            if (Float.compare(result.resultSimilarityMaximum(), jPlagComparison.maximalSimilarity()) != 0) {
+                addToValidationErrors("maximalSimilarity", String.valueOf(result.resultSimilarityMaximum()),
+                        String.valueOf(jPlagComparison.maximalSimilarity()));
+            }
+            if (Integer.compare(result.resultMatchedTokenNumber(), jPlagComparison.getNumberOfMatchedTokens()) != 0) {
+                addToValidationErrors("numberOfMatchedTokens", String.valueOf(result.resultMatchedTokenNumber()),
+                        String.valueOf(jPlagComparison.getNumberOfMatchedTokens()));
+            }
+
+            assertTrue(validationErrors.isEmpty(), createValidationErrorOutout());
+        }
+    }
+
+    /**
+     * Creates the display message for failed tests
+     * @param valueName Name of the failed test object
+     * @param currentValue current test values
+     * @param expectedValue expected test values
+     */
+    private void addToValidationErrors(String valueName, String currentValue, String expectedValue) {
+        validationErrors.add(valueName + " was " + currentValue + " but expected " + expectedValue);
+    }
+
+    /**
+     * Creates the display info from the current failed test results
+     * @return formatted text for the failed comparative values of the current test
+     */
+    private String createValidationErrorOutout() {
+        StringJoiner joiner = new StringJoiner(System.lineSeparator());
+        joiner.add(""); // empty line at start
+        joiner.add("There were <" + validationErrors.size() + "> validation error(s):");
+
+        validationErrors.stream().forEach(errorLine -> joiner.add(errorLine));
+        joiner.add(""); // line break at the end
+
+        return joiner.toString();
+    }
+
+    /**
+     * Add or create the current test results in a temporary list to be able to save them later.
+     * @param directoryName name of the current tested directory
+     * @param options for the current test run
+     * @param jPlagComparison current test results
+     * @param languageIdentifier current JPlag language option
+     */
+    private void addToTemporaryResultMap(String directoryName, Options options, JPlagComparison jPlagComparison, String languageIdentifier) {
+        var element = temporaryResultList.get(directoryName);
+
+        if (element != null) {
+            for (var item : element) {
+                if (item.options().equals(options)) {
+                    item.putIdentifierToResultMap(TestSuiteHelper.getTestIdentifier(jPlagComparison), new ExpectedResult(
+                            jPlagComparison.minimalSimilarity(), jPlagComparison.maximalSimilarity(), jPlagComparison.getNumberOfMatchedTokens()));
+                    return;
+                }
+            }
+            Map<String, ExpectedResult> temporaryHashMap = new HashMap<>();
+            temporaryHashMap.put(TestSuiteHelper.getTestIdentifier(jPlagComparison), new ExpectedResult(jPlagComparison.minimalSimilarity(),
+                    jPlagComparison.maximalSimilarity(), jPlagComparison.getNumberOfMatchedTokens()));
+            element.add(new ResultDescription(languageIdentifier, options, temporaryHashMap));
+        } else {
+            var temporaryNewResultList = new ArrayList<ResultDescription>();
+            Map<String, ExpectedResult> temporaryHashMap = new HashMap<>();
+            temporaryHashMap.put(TestSuiteHelper.getTestIdentifier(jPlagComparison), new ExpectedResult(jPlagComparison.minimalSimilarity(),
+                    jPlagComparison.maximalSimilarity(), jPlagComparison.getNumberOfMatchedTokens()));
+
+            temporaryNewResultList.add(new ResultDescription(languageIdentifier, options, temporaryHashMap));
+
+            temporaryResultList.put(directoryName, temporaryNewResultList);
+        }
+    }
+
+    /**
+     * Creates the name of the test for better assignment and readability Pattern: Language: (option values)
+     * filename1-filename2
+     * @param option under which the current test run
+     * @param languageIdentifier current language used in the test
+     * @param testFiles test data for assigning by filename
+     * @return display name for the individual tests
+     */
+    private String getTestCaseDisplayName(Options option, String languageIdentifier, String[] testFiles) {
+        StringBuilder stringBuilder = new StringBuilder();
+        stringBuilder.append("(" + String.valueOf(option.minimumTokenMatch()) + ")");
+        for (int counter = 0; counter < testFiles.length; counter++) {
+            String fileName = Path.of(testFiles[counter]).toFile().getName();
+            stringBuilder.append(fileName.substring(0, fileName.lastIndexOf('.')));
+            if (counter + 1 < testFiles.length) {
+                stringBuilder.append("-");
+            }
+        }
+        return languageIdentifier + ": " + stringBuilder.toString();
+    }
+}

+ 0 - 114
jplag.endToEndTesting/src/test/java/de/jplag/end_to_end_testing/JavaEndToEndTest.java

@@ -1,114 +0,0 @@
-package de.jplag.end_to_end_testing;
-
-import static org.junit.jupiter.api.Assertions.assertEquals;
-import static org.junit.jupiter.api.Assertions.assertTrue;
-
-import java.io.IOException;
-import java.util.stream.Stream;
-
-import org.junit.jupiter.api.AfterEach;
-import org.junit.jupiter.api.BeforeAll;
-import org.junit.jupiter.api.TestInstance;
-import org.junit.jupiter.params.ParameterizedTest;
-import org.junit.jupiter.params.provider.Arguments;
-import org.junit.jupiter.params.provider.MethodSource;
-
-import de.jplag.JPlag;
-import de.jplag.JPlagComparison;
-import de.jplag.JPlagResult;
-import de.jplag.end_to_end_testing.constants.TestDirectoryConstants;
-import de.jplag.end_to_end_testing.helper.JPlagTestSuiteHelper;
-import de.jplag.end_to_end_testing.model.TestCaseModel;
-import de.jplag.exceptions.ExitException;
-import de.jplag.java.Language;
-
-/**
- * Main test class for end-to-end testing in the Java language. The test cases aim to detect changes in the detection of
- * plagiarism in the Java language and to be able to roughly categorize them. The plagiarism is compared with the
- * original class. The results are compared with the results from previous tests and changes are detected.
- */
-@TestInstance(TestInstance.Lifecycle.PER_CLASS)
-class JavaEndToEndTest {
-
-    private JPlagTestSuiteHelper jplagTestSuiteHelper;
-
-    @BeforeAll
-    public void setUp() throws IOException {
-        jplagTestSuiteHelper = new JPlagTestSuiteHelper(Language.IDENTIFIER);
-        assertTrue(TestDirectoryConstants.BASE_PATH_TO_JAVA_RESOURCES_SORTALGO.toFile().exists(), "Could not find base directory!");
-        assertTrue(TestDirectoryConstants.BASE_PATH_TO_JAVA_RESULT_JSON.toFile().isFile(), "Could not find java result json!");
-    }
-
-    @AfterEach
-    public void teardown() throws IOException {
-        // after close the created directories are deleted
-        jplagTestSuiteHelper.clear();
-    }
-
-    /**
-     * Test cases created for the normalization level
-     * @return the classes to be tested with the corresponding identifier
-     */
-    private static Stream<Arguments> normalizationLevelTestArguments() {
-        return Stream.of(Arguments.of((Object) new String[] {"SortAlgo.java", "SortAlgo1.java"}, 0),
-                Arguments.of((Object) new String[] {"SortAlgo.java", "SortAlgo2.java"}, 1),
-                Arguments.of((Object) new String[] {"SortAlgo1.java", "SortAlgo2.java"}, 2));
-    }
-
-    /**
-     * Test cases for the token generation level
-     * @return the classes to be tested with the corresponding identifier
-     */
-    private static Stream<Arguments> tokenGenerationLevelTestArguments() {
-        return Stream.of(Arguments.of((Object) new String[] {"SortAlgo.java", "SortAlgo3.java"}, 0),
-                Arguments.of((Object) new String[] {"SortAlgo.java", "SortAlgo4.java"}, 1),
-                Arguments.of((Object) new String[] {"SortAlgo.java", "SortAlgo4d1.java"}, 2));
-    }
-
-    /**
-     * Inserting comments or empty lines (normalization level) -> id = 0 Changing variable names or function names
-     * (normalization level) -> id = 1 Inserting comments or empty lines (normalization level) -> id = 2
-     * @param testClassNames Plagiarized classes names in the resource directorie which are needed for the test
-     * @param testIdentifier name of the testId to load and identify the stored results
-     * @throws IOException is thrown in case of problems with copying the plagiarism classes
-     * @throws ExitException in case the plagiarism detection with JPlag is preemptively terminated would be of the test.
-     */
-    @ParameterizedTest
-    @MethodSource("normalizationLevelTestArguments")
-    void normalizationLevelTest(String[] testClassNames, int testIdentifier) throws IOException, ExitException {
-        runJPlagTestSuite(testClassNames, testIdentifier);
-    }
-
-    /**
-     * Insertion of unnecessary or changed code lines (token generation) -> id = 0 Changing the program flow (token
-     * generation) (statements and functions must be independent from each other) -> id = 1 Variable declaration at the
-     * beginning of the program (Detecting Source Code Plagiarism [...]) -> id = 2
-     * @param testClassNames Plagiarized classes names in the resource directorie which are needed for the test
-     * @param testIdentifier name of the testId to load and identify the stored results
-     * @throws IOException is thrown in case of problems with copying the plagiarism classes
-     * @throws ExitException in case the plagiarism detection with JPlag is preemptively terminated would be of the test.
-     */
-    @ParameterizedTest
-    @MethodSource("tokenGenerationLevelTestArguments")
-    void tokenGenerationLevelTest(String[] testClassNames, int testIdentifier) throws IOException, ExitException {
-        runJPlagTestSuite(testClassNames, testIdentifier);
-    }
-
-    /**
-     * This method creates the necessary results as well as models for a test run and summarizes them for a comparison.
-     * @param testClassNames Plagiarized classes names in the resource directorie which are needed for the test
-     * @param testIdentifier name of the testId to load and identify the stored results
-     * @throws IOException is thrown in case of problems with copying the plagiarism classes
-     * @throws ExitException in case the plagiarism detection with JPlag is preemptively terminated would be of the test.
-     */
-    private void runJPlagTestSuite(String[] testClassNames, int testIdentifier) throws IOException, ExitException {
-        String functionName = StackWalker.getInstance().walk(stream -> stream.skip(1).findFirst().get()).getMethodName();
-        TestCaseModel testCaseModel = jplagTestSuiteHelper.createNewTestCase(testClassNames, functionName);
-        JPlagResult jplagResult = new JPlag(testCaseModel.getJPlagOptionsFromCurrentModel()).run();
-
-        for (JPlagComparison jPlagComparison : jplagResult.getAllComparisons()) {
-            assertEquals(testCaseModel.getCurrentJsonModel().getResultModelById(testIdentifier).getResultSimilarity(), jPlagComparison.similarity(),
-                    "The JPlag results [similarity] do not match the stored values!");
-        }
-    }
-}

+ 0 - 0
jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo.java → jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo.java


+ 0 - 0
jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo1.java → jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo1.java


+ 0 - 0
jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo1_2.java → jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo1_2.java


+ 0 - 0
jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo1_3.java → jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo1_3.java


+ 0 - 0
jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo1_4.java → jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo1_4.java


+ 0 - 0
jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo1_5.java → jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo1_5.java


+ 0 - 0
jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo1_6.java → jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo1_6.java


+ 0 - 0
jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo2.java → jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo2.java


+ 0 - 0
jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo2_5.java → jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo2_5.java


+ 0 - 0
jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo3.java → jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo3.java


+ 0 - 0
jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo3_5.java → jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo3_5.java


+ 0 - 0
jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo3_6.java → jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo3_6.java


+ 0 - 0
jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo4.java → jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo4.java


+ 0 - 0
jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo4d1.java → jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo4d1.java


+ 0 - 0
jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo4d2.java → jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo4d2.java


+ 0 - 0
jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo4d3.java → jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo4d3.java


+ 0 - 0
jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo5.java → jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo5.java


+ 0 - 0
jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo6.java → jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo6.java


+ 0 - 0
jplag.endToEndTesting/src/test/resources/java/sortAlgo/SortAlgo7.java → jplag.endToEndTesting/src/test/resources/languageTestFiles/java/sortAlgo/SortAlgo7.java


+ 0 - 36
jplag.endToEndTesting/src/test/resources/results/JavaResult.json

@@ -1,36 +0,0 @@
-[
-  {
-    "function_name": "normalizationLevelTest",
-    "test_results": [
-      {
-        "result_similarity": 100,
-        "test_identifier": 0
-      },
-      {
-        "result_similarity": 100,
-        "test_identifier": 1
-      },
-      {
-        "result_similarity": 100,
-        "test_identifier": 2
-      }
-    ]
-  },
-  {
-    "function_name": "tokenGenerationLevelTest",
-    "test_results": [
-      {
-        "result_similarity": 64.0,
-        "test_identifier": 0
-      },
-      {
-        "result_similarity": 73.04348,
-        "test_identifier": 1
-      },
-      {
-        "result_similarity": 73.94958,
-        "test_identifier": 2
-      }
-    ]
-  }
-]

+ 1723 - 0
jplag.endToEndTesting/src/test/resources/results/java/sortAlgo.json

@@ -0,0 +1,1723 @@
+[ {
+  "options" : {
+    "minimum_token_match" : 1
+  },
+  "tests" : {
+    "SortAlgo-SortAlgo5" : {
+      "minimal_similarity" : 82.14286,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo-SortAlgo6" : {
+      "minimal_similarity" : 83.58209,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo-SortAlgo7" : {
+      "minimal_similarity" : 96.42857,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 54
+    },
+    "SortAlgo3_5-SortAlgo3_6" : {
+      "minimal_similarity" : 78.666664,
+      "maximum_similarity" : 85.50725,
+      "matched_token_number" : 59
+    },
+    "SortAlgo6-SortAlgo7" : {
+      "minimal_similarity" : 80.597015,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 54
+    },
+    "SortAlgo1_3-SortAlgo7" : {
+      "minimal_similarity" : 72.0,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 54
+    },
+    "SortAlgo1_3-SortAlgo6" : {
+      "minimal_similarity" : 85.333336,
+      "maximum_similarity" : 95.522385,
+      "matched_token_number" : 64
+    },
+    "SortAlgo-SortAlgo1" : {
+      "minimal_similarity" : 100.0,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_3-SortAlgo3" : {
+      "minimal_similarity" : 92.0,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 69
+    },
+    "SortAlgo-SortAlgo2" : {
+      "minimal_similarity" : 100.0,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_3-SortAlgo2" : {
+      "minimal_similarity" : 74.666664,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo-SortAlgo3" : {
+      "minimal_similarity" : 81.159424,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_3-SortAlgo5" : {
+      "minimal_similarity" : 62.666668,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo-SortAlgo4" : {
+      "minimal_similarity" : 94.91525,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_3-SortAlgo4" : {
+      "minimal_similarity" : 77.333336,
+      "maximum_similarity" : 98.305084,
+      "matched_token_number" : 58
+    },
+    "SortAlgo2-SortAlgo2_5" : {
+      "minimal_similarity" : 82.14286,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo1_2-SortAlgo2" : {
+      "minimal_similarity" : 73.68421,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_2-SortAlgo3" : {
+      "minimal_similarity" : 82.89474,
+      "maximum_similarity" : 91.304344,
+      "matched_token_number" : 63
+    },
+    "SortAlgo5-SortAlgo6" : {
+      "minimal_similarity" : 70.149254,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo1_2-SortAlgo4" : {
+      "minimal_similarity" : 76.31579,
+      "maximum_similarity" : 98.305084,
+      "matched_token_number" : 58
+    },
+    "SortAlgo5-SortAlgo7" : {
+      "minimal_similarity" : 82.14286,
+      "maximum_similarity" : 85.18519,
+      "matched_token_number" : 46
+    },
+    "SortAlgo1_2-SortAlgo5" : {
+      "minimal_similarity" : 61.842106,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo1_2-SortAlgo6" : {
+      "minimal_similarity" : 84.210526,
+      "maximum_similarity" : 95.522385,
+      "matched_token_number" : 64
+    },
+    "SortAlgo1_2-SortAlgo7" : {
+      "minimal_similarity" : 71.052635,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 54
+    },
+    "SortAlgo1-SortAlgo2_5" : {
+      "minimal_similarity" : 82.14286,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo1_3-SortAlgo3_6" : {
+      "minimal_similarity" : 93.333336,
+      "maximum_similarity" : 93.333336,
+      "matched_token_number" : 70
+    },
+    "SortAlgo4-SortAlgo5" : {
+      "minimal_similarity" : 77.9661,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo4-SortAlgo6" : {
+      "minimal_similarity" : 88.0597,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 59
+    },
+    "SortAlgo1_4-SortAlgo3_5" : {
+      "minimal_similarity" : 72.46377,
+      "maximum_similarity" : 84.745766,
+      "matched_token_number" : 50
+    },
+    "SortAlgo4-SortAlgo7" : {
+      "minimal_similarity" : 91.52542,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 54
+    },
+    "SortAlgo1_4-SortAlgo3_6" : {
+      "minimal_similarity" : 78.666664,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 59
+    },
+    "SortAlgo1_5-SortAlgo3_6" : {
+      "minimal_similarity" : 62.666668,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo1_5-SortAlgo3_5" : {
+      "minimal_similarity" : 81.159424,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_6-SortAlgo3_5" : {
+      "minimal_similarity" : 73.91304,
+      "maximum_similarity" : 85.0,
+      "matched_token_number" : 51
+    },
+    "SortAlgo-SortAlgo4d3" : {
+      "minimal_similarity" : 90.32258,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_6-SortAlgo3_6" : {
+      "minimal_similarity" : 80.0,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 60
+    },
+    "SortAlgo-SortAlgo4d2" : {
+      "minimal_similarity" : 87.5,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo-SortAlgo4d1" : {
+      "minimal_similarity" : 87.30159,
+      "maximum_similarity" : 98.21429,
+      "matched_token_number" : 55
+    },
+    "SortAlgo1_4-SortAlgo3" : {
+      "minimal_similarity" : 84.05797,
+      "maximum_similarity" : 98.305084,
+      "matched_token_number" : 58
+    },
+    "SortAlgo3-SortAlgo3_5" : {
+      "minimal_similarity" : 85.50725,
+      "maximum_similarity" : 85.50725,
+      "matched_token_number" : 59
+    },
+    "SortAlgo1_4-SortAlgo4" : {
+      "minimal_similarity" : 100.0,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 59
+    },
+    "SortAlgo1_4-SortAlgo2" : {
+      "minimal_similarity" : 94.91525,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo3-SortAlgo3_6" : {
+      "minimal_similarity" : 92.0,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 69
+    },
+    "SortAlgo1_4-SortAlgo7" : {
+      "minimal_similarity" : 91.52542,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 54
+    },
+    "SortAlgo1_4-SortAlgo5" : {
+      "minimal_similarity" : 77.9661,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo2-SortAlgo3_5" : {
+      "minimal_similarity" : 72.46377,
+      "maximum_similarity" : 89.28571,
+      "matched_token_number" : 50
+    },
+    "SortAlgo1_4-SortAlgo6" : {
+      "minimal_similarity" : 88.0597,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 59
+    },
+    "SortAlgo2-SortAlgo3_6" : {
+      "minimal_similarity" : 74.666664,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1-SortAlgo3_6" : {
+      "minimal_similarity" : 74.666664,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1-SortAlgo3_5" : {
+      "minimal_similarity" : 72.46377,
+      "maximum_similarity" : 89.28571,
+      "matched_token_number" : 50
+    },
+    "SortAlgo-SortAlgo1_2" : {
+      "minimal_similarity" : 73.68421,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_3-SortAlgo3_5" : {
+      "minimal_similarity" : 80.0,
+      "maximum_similarity" : 86.95652,
+      "matched_token_number" : 60
+    },
+    "SortAlgo1_2-SortAlgo3_5" : {
+      "minimal_similarity" : 72.36842,
+      "maximum_similarity" : 79.710144,
+      "matched_token_number" : 55
+    },
+    "SortAlgo1_2-SortAlgo3_6" : {
+      "minimal_similarity" : 88.1579,
+      "maximum_similarity" : 89.333336,
+      "matched_token_number" : 67
+    },
+    "SortAlgo-SortAlgo1_6" : {
+      "minimal_similarity" : 93.333336,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo-SortAlgo1_5" : {
+      "minimal_similarity" : 82.14286,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo-SortAlgo1_4" : {
+      "minimal_similarity" : 94.91525,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo-SortAlgo1_3" : {
+      "minimal_similarity" : 74.666664,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_6-SortAlgo4d1" : {
+      "minimal_similarity" : 88.888885,
+      "maximum_similarity" : 93.333336,
+      "matched_token_number" : 56
+    },
+    "SortAlgo4d2-SortAlgo7" : {
+      "minimal_similarity" : 84.375,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 54
+    },
+    "SortAlgo1_6-SortAlgo4d2" : {
+      "minimal_similarity" : 89.0625,
+      "maximum_similarity" : 95.0,
+      "matched_token_number" : 57
+    },
+    "SortAlgo4d2-SortAlgo5" : {
+      "minimal_similarity" : 73.4375,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo4d2-SortAlgo6" : {
+      "minimal_similarity" : 89.55224,
+      "maximum_similarity" : 93.75,
+      "matched_token_number" : 60
+    },
+    "SortAlgo1_6-SortAlgo4d3" : {
+      "minimal_similarity" : 91.935486,
+      "maximum_similarity" : 95.0,
+      "matched_token_number" : 57
+    },
+    "SortAlgo1_5-SortAlgo2_5" : {
+      "minimal_similarity" : 100.0,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_2-SortAlgo4d1" : {
+      "minimal_similarity" : 76.31579,
+      "maximum_similarity" : 92.06349,
+      "matched_token_number" : 58
+    },
+    "SortAlgo1_3-SortAlgo4d3" : {
+      "minimal_similarity" : 82.666664,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 62
+    },
+    "SortAlgo2-SortAlgo3" : {
+      "minimal_similarity" : 81.159424,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_2-SortAlgo4d2" : {
+      "minimal_similarity" : 77.63158,
+      "maximum_similarity" : 92.1875,
+      "matched_token_number" : 59
+    },
+    "SortAlgo1_3-SortAlgo4d2" : {
+      "minimal_similarity" : 85.333336,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 64
+    },
+    "SortAlgo2-SortAlgo4" : {
+      "minimal_similarity" : 94.91525,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_6-SortAlgo2_5" : {
+      "minimal_similarity" : 76.666664,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo1_4-SortAlgo4d1" : {
+      "minimal_similarity" : 87.30159,
+      "maximum_similarity" : 93.22034,
+      "matched_token_number" : 55
+    },
+    "SortAlgo1_5-SortAlgo4d3" : {
+      "minimal_similarity" : 75.80645,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo1_4-SortAlgo4d2" : {
+      "minimal_similarity" : 87.5,
+      "maximum_similarity" : 94.91525,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_5-SortAlgo4d2" : {
+      "minimal_similarity" : 73.4375,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo1_4-SortAlgo4d3" : {
+      "minimal_similarity" : 90.32258,
+      "maximum_similarity" : 94.91525,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_5-SortAlgo4d1" : {
+      "minimal_similarity" : 73.01588,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo4d2-SortAlgo4d3" : {
+      "minimal_similarity" : 96.875,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 62
+    },
+    "SortAlgo2-SortAlgo7" : {
+      "minimal_similarity" : 96.42857,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 54
+    },
+    "SortAlgo1_2-SortAlgo4d3" : {
+      "minimal_similarity" : 76.31579,
+      "maximum_similarity" : 93.548386,
+      "matched_token_number" : 58
+    },
+    "SortAlgo1_3-SortAlgo4d1" : {
+      "minimal_similarity" : 84.0,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 63
+    },
+    "SortAlgo2-SortAlgo5" : {
+      "minimal_similarity" : 82.14286,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo4d1-SortAlgo4d2" : {
+      "minimal_similarity" : 98.4375,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 63
+    },
+    "SortAlgo2-SortAlgo6" : {
+      "minimal_similarity" : 83.58209,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo4d1-SortAlgo4d3" : {
+      "minimal_similarity" : 96.82539,
+      "maximum_similarity" : 98.3871,
+      "matched_token_number" : 61
+    },
+    "SortAlgo1_6-SortAlgo7" : {
+      "minimal_similarity" : 90.0,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 54
+    },
+    "SortAlgo2_5-SortAlgo3_5" : {
+      "minimal_similarity" : 81.159424,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo3_6-SortAlgo6" : {
+      "minimal_similarity" : 89.333336,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 67
+    },
+    "SortAlgo2_5-SortAlgo3_6" : {
+      "minimal_similarity" : 62.666668,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo3_6-SortAlgo5" : {
+      "minimal_similarity" : 62.666668,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo3_6-SortAlgo7" : {
+      "minimal_similarity" : 72.0,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 54
+    },
+    "SortAlgo4d3-SortAlgo6" : {
+      "minimal_similarity" : 89.55224,
+      "maximum_similarity" : 96.77419,
+      "matched_token_number" : 60
+    },
+    "SortAlgo4d3-SortAlgo7" : {
+      "minimal_similarity" : 87.09677,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 54
+    },
+    "SortAlgo4d3-SortAlgo5" : {
+      "minimal_similarity" : 75.80645,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo1_4-SortAlgo2_5" : {
+      "minimal_similarity" : 77.9661,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo1-SortAlgo4" : {
+      "minimal_similarity" : 94.91525,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1-SortAlgo5" : {
+      "minimal_similarity" : 82.14286,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo1-SortAlgo2" : {
+      "minimal_similarity" : 100.0,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo3_6-SortAlgo4" : {
+      "minimal_similarity" : 78.666664,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 59
+    },
+    "SortAlgo1-SortAlgo3" : {
+      "minimal_similarity" : 81.159424,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_3-SortAlgo2_5" : {
+      "minimal_similarity" : 62.666668,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo1_6-SortAlgo2" : {
+      "minimal_similarity" : 93.333336,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1-SortAlgo6" : {
+      "minimal_similarity" : 83.58209,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1-SortAlgo7" : {
+      "minimal_similarity" : 96.42857,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 54
+    },
+    "SortAlgo1_2-SortAlgo2_5" : {
+      "minimal_similarity" : 61.842106,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo-SortAlgo2_5" : {
+      "minimal_similarity" : 82.14286,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo1_6-SortAlgo6" : {
+      "minimal_similarity" : 89.55224,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 60
+    },
+    "SortAlgo1_6-SortAlgo5" : {
+      "minimal_similarity" : 76.666664,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo1_6-SortAlgo4" : {
+      "minimal_similarity" : 98.333336,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 59
+    },
+    "SortAlgo1_6-SortAlgo3" : {
+      "minimal_similarity" : 85.50725,
+      "maximum_similarity" : 98.333336,
+      "matched_token_number" : 59
+    },
+    "SortAlgo1_5-SortAlgo6" : {
+      "minimal_similarity" : 70.149254,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo1_5-SortAlgo7" : {
+      "minimal_similarity" : 82.14286,
+      "maximum_similarity" : 85.18519,
+      "matched_token_number" : 46
+    },
+    "SortAlgo2_5-SortAlgo7" : {
+      "minimal_similarity" : 82.14286,
+      "maximum_similarity" : 85.18519,
+      "matched_token_number" : 46
+    },
+    "SortAlgo1-SortAlgo4d3" : {
+      "minimal_similarity" : 90.32258,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_5-SortAlgo4" : {
+      "minimal_similarity" : 77.9661,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo1-SortAlgo4d2" : {
+      "minimal_similarity" : 87.5,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_5-SortAlgo5" : {
+      "minimal_similarity" : 100.0,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1-SortAlgo4d1" : {
+      "minimal_similarity" : 87.30159,
+      "maximum_similarity" : 98.21429,
+      "matched_token_number" : 55
+    },
+    "SortAlgo1_5-SortAlgo2" : {
+      "minimal_similarity" : 82.14286,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo3_5-SortAlgo6" : {
+      "minimal_similarity" : 81.159424,
+      "maximum_similarity" : 83.58209,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_5-SortAlgo3" : {
+      "minimal_similarity" : 68.115944,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo3_5-SortAlgo7" : {
+      "minimal_similarity" : 69.565216,
+      "maximum_similarity" : 88.888885,
+      "matched_token_number" : 48
+    },
+    "SortAlgo3-SortAlgo4d1" : {
+      "minimal_similarity" : 88.4058,
+      "maximum_similarity" : 96.82539,
+      "matched_token_number" : 61
+    },
+    "SortAlgo3-SortAlgo4d3" : {
+      "minimal_similarity" : 86.95652,
+      "maximum_similarity" : 96.77419,
+      "matched_token_number" : 60
+    },
+    "SortAlgo3-SortAlgo4d2" : {
+      "minimal_similarity" : 89.85507,
+      "maximum_similarity" : 96.875,
+      "matched_token_number" : 62
+    },
+    "SortAlgo2-SortAlgo4d3" : {
+      "minimal_similarity" : 90.32258,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo2-SortAlgo4d1" : {
+      "minimal_similarity" : 87.30159,
+      "maximum_similarity" : 98.21429,
+      "matched_token_number" : 55
+    },
+    "SortAlgo1_5-SortAlgo1_6" : {
+      "minimal_similarity" : 76.666664,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo2-SortAlgo4d2" : {
+      "minimal_similarity" : 87.5,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo3_6-SortAlgo4d2" : {
+      "minimal_similarity" : 84.0,
+      "maximum_similarity" : 98.4375,
+      "matched_token_number" : 63
+    },
+    "SortAlgo4d1-SortAlgo7" : {
+      "minimal_similarity" : 84.12698,
+      "maximum_similarity" : 98.14815,
+      "matched_token_number" : 53
+    },
+    "SortAlgo3_6-SortAlgo4d1" : {
+      "minimal_similarity" : 82.666664,
+      "maximum_similarity" : 98.4127,
+      "matched_token_number" : 62
+    },
+    "SortAlgo4d1-SortAlgo6" : {
+      "minimal_similarity" : 88.0597,
+      "maximum_similarity" : 93.650795,
+      "matched_token_number" : 59
+    },
+    "SortAlgo4-SortAlgo4d1" : {
+      "minimal_similarity" : 87.30159,
+      "maximum_similarity" : 93.22034,
+      "matched_token_number" : 55
+    },
+    "SortAlgo3_6-SortAlgo4d3" : {
+      "minimal_similarity" : 81.333336,
+      "maximum_similarity" : 98.3871,
+      "matched_token_number" : 61
+    },
+    "SortAlgo4-SortAlgo4d2" : {
+      "minimal_similarity" : 87.5,
+      "maximum_similarity" : 94.91525,
+      "matched_token_number" : 56
+    },
+    "SortAlgo4-SortAlgo4d3" : {
+      "minimal_similarity" : 90.32258,
+      "maximum_similarity" : 94.91525,
+      "matched_token_number" : 56
+    },
+    "SortAlgo3_5-SortAlgo4" : {
+      "minimal_similarity" : 72.46377,
+      "maximum_similarity" : 84.745766,
+      "matched_token_number" : 50
+    },
+    "SortAlgo3_5-SortAlgo4d2" : {
+      "minimal_similarity" : 81.159424,
+      "maximum_similarity" : 87.5,
+      "matched_token_number" : 56
+    },
+    "SortAlgo4d1-SortAlgo5" : {
+      "minimal_similarity" : 73.01588,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo3_5-SortAlgo4d3" : {
+      "minimal_similarity" : 78.26087,
+      "maximum_similarity" : 87.09677,
+      "matched_token_number" : 54
+    },
+    "SortAlgo3_5-SortAlgo5" : {
+      "minimal_similarity" : 81.159424,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo2_5-SortAlgo4" : {
+      "minimal_similarity" : 77.9661,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo2_5-SortAlgo3" : {
+      "minimal_similarity" : 68.115944,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo3_5-SortAlgo4d1" : {
+      "minimal_similarity" : 79.710144,
+      "maximum_similarity" : 87.30159,
+      "matched_token_number" : 55
+    },
+    "SortAlgo2_5-SortAlgo6" : {
+      "minimal_similarity" : 70.149254,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo2_5-SortAlgo5" : {
+      "minimal_similarity" : 100.0,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1-SortAlgo1_3" : {
+      "minimal_similarity" : 74.666664,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1-SortAlgo1_2" : {
+      "minimal_similarity" : 73.68421,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1-SortAlgo1_6" : {
+      "minimal_similarity" : 93.333336,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1-SortAlgo1_5" : {
+      "minimal_similarity" : 82.14286,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo1-SortAlgo1_4" : {
+      "minimal_similarity" : 94.91525,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo2_5-SortAlgo4d1" : {
+      "minimal_similarity" : 73.01588,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo3-SortAlgo7" : {
+      "minimal_similarity" : 78.26087,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 54
+    },
+    "SortAlgo2_5-SortAlgo4d2" : {
+      "minimal_similarity" : 73.4375,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo3-SortAlgo6" : {
+      "minimal_similarity" : 92.753624,
+      "maximum_similarity" : 95.522385,
+      "matched_token_number" : 64
+    },
+    "SortAlgo1_4-SortAlgo1_5" : {
+      "minimal_similarity" : 77.9661,
+      "maximum_similarity" : 82.14286,
+      "matched_token_number" : 46
+    },
+    "SortAlgo3-SortAlgo5" : {
+      "minimal_similarity" : 68.115944,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo1_4-SortAlgo1_6" : {
+      "minimal_similarity" : 98.333336,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 59
+    },
+    "SortAlgo3-SortAlgo4" : {
+      "minimal_similarity" : 84.05797,
+      "maximum_similarity" : 98.305084,
+      "matched_token_number" : 58
+    },
+    "SortAlgo1_3-SortAlgo1_5" : {
+      "minimal_similarity" : 62.666668,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo1_3-SortAlgo1_4" : {
+      "minimal_similarity" : 77.333336,
+      "maximum_similarity" : 98.305084,
+      "matched_token_number" : 58
+    },
+    "SortAlgo2_5-SortAlgo4d3" : {
+      "minimal_similarity" : 75.80645,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo1_3-SortAlgo1_6" : {
+      "minimal_similarity" : 78.666664,
+      "maximum_similarity" : 98.333336,
+      "matched_token_number" : 59
+    },
+    "SortAlgo-SortAlgo3_6" : {
+      "minimal_similarity" : 74.666664,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_2-SortAlgo1_5" : {
+      "minimal_similarity" : 61.842106,
+      "maximum_similarity" : 83.92857,
+      "matched_token_number" : 47
+    },
+    "SortAlgo-SortAlgo3_5" : {
+      "minimal_similarity" : 72.46377,
+      "maximum_similarity" : 89.28571,
+      "matched_token_number" : 50
+    },
+    "SortAlgo1_2-SortAlgo1_6" : {
+      "minimal_similarity" : 76.31579,
+      "maximum_similarity" : 96.666664,
+      "matched_token_number" : 58
+    },
+    "SortAlgo1_2-SortAlgo1_3" : {
+      "minimal_similarity" : 82.89474,
+      "maximum_similarity" : 84.0,
+      "matched_token_number" : 63
+    },
+    "SortAlgo1_2-SortAlgo1_4" : {
+      "minimal_similarity" : 76.31579,
+      "maximum_similarity" : 98.305084,
+      "matched_token_number" : 58
+    }
+  }
+}, {
+  "options" : {
+    "minimum_token_match" : 15
+  },
+  "tests" : {
+    "SortAlgo-SortAlgo5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo-SortAlgo6" : {
+      "minimal_similarity" : 43.28358,
+      "maximum_similarity" : 51.785713,
+      "matched_token_number" : 29
+    },
+    "SortAlgo-SortAlgo7" : {
+      "minimal_similarity" : 82.14286,
+      "maximum_similarity" : 85.18519,
+      "matched_token_number" : 46
+    },
+    "SortAlgo3_5-SortAlgo3_6" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo6-SortAlgo7" : {
+      "minimal_similarity" : 43.28358,
+      "maximum_similarity" : 53.703705,
+      "matched_token_number" : 29
+    },
+    "SortAlgo1_3-SortAlgo7" : {
+      "minimal_similarity" : 53.333332,
+      "maximum_similarity" : 74.07407,
+      "matched_token_number" : 40
+    },
+    "SortAlgo1_3-SortAlgo6" : {
+      "minimal_similarity" : 32.0,
+      "maximum_similarity" : 35.820896,
+      "matched_token_number" : 24
+    },
+    "SortAlgo-SortAlgo1" : {
+      "minimal_similarity" : 100.0,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_3-SortAlgo3" : {
+      "minimal_similarity" : 54.666668,
+      "maximum_similarity" : 59.420288,
+      "matched_token_number" : 41
+    },
+    "SortAlgo-SortAlgo2" : {
+      "minimal_similarity" : 100.0,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_3-SortAlgo2" : {
+      "minimal_similarity" : 53.333332,
+      "maximum_similarity" : 71.42857,
+      "matched_token_number" : 40
+    },
+    "SortAlgo-SortAlgo3" : {
+      "minimal_similarity" : 57.971016,
+      "maximum_similarity" : 71.42857,
+      "matched_token_number" : 40
+    },
+    "SortAlgo1_3-SortAlgo5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo-SortAlgo4" : {
+      "minimal_similarity" : 35.59322,
+      "maximum_similarity" : 37.5,
+      "matched_token_number" : 21
+    },
+    "SortAlgo1_3-SortAlgo4" : {
+      "minimal_similarity" : 24.0,
+      "maximum_similarity" : 30.508474,
+      "matched_token_number" : 18
+    },
+    "SortAlgo2-SortAlgo2_5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_2-SortAlgo2" : {
+      "minimal_similarity" : 59.210526,
+      "maximum_similarity" : 80.35714,
+      "matched_token_number" : 45
+    },
+    "SortAlgo1_2-SortAlgo3" : {
+      "minimal_similarity" : 52.63158,
+      "maximum_similarity" : 57.971016,
+      "matched_token_number" : 40
+    },
+    "SortAlgo5-SortAlgo6" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_2-SortAlgo4" : {
+      "minimal_similarity" : 28.947369,
+      "maximum_similarity" : 37.288136,
+      "matched_token_number" : 22
+    },
+    "SortAlgo5-SortAlgo7" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_2-SortAlgo5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_2-SortAlgo6" : {
+      "minimal_similarity" : 38.157894,
+      "maximum_similarity" : 43.28358,
+      "matched_token_number" : 29
+    },
+    "SortAlgo1_2-SortAlgo7" : {
+      "minimal_similarity" : 46.05263,
+      "maximum_similarity" : 64.81481,
+      "matched_token_number" : 35
+    },
+    "SortAlgo1-SortAlgo2_5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_3-SortAlgo3_6" : {
+      "minimal_similarity" : 32.0,
+      "maximum_similarity" : 32.0,
+      "matched_token_number" : 24
+    },
+    "SortAlgo4-SortAlgo5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo4-SortAlgo6" : {
+      "minimal_similarity" : 31.343283,
+      "maximum_similarity" : 35.59322,
+      "matched_token_number" : 21
+    },
+    "SortAlgo1_4-SortAlgo3_5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo4-SortAlgo7" : {
+      "minimal_similarity" : 35.59322,
+      "maximum_similarity" : 38.88889,
+      "matched_token_number" : 21
+    },
+    "SortAlgo1_4-SortAlgo3_6" : {
+      "minimal_similarity" : 32.0,
+      "maximum_similarity" : 40.677967,
+      "matched_token_number" : 24
+    },
+    "SortAlgo1_5-SortAlgo3_6" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_5-SortAlgo3_5" : {
+      "minimal_similarity" : 59.420288,
+      "maximum_similarity" : 73.21429,
+      "matched_token_number" : 41
+    },
+    "SortAlgo1_6-SortAlgo3_5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo-SortAlgo4d3" : {
+      "minimal_similarity" : 70.96774,
+      "maximum_similarity" : 78.57143,
+      "matched_token_number" : 44
+    },
+    "SortAlgo1_6-SortAlgo3_6" : {
+      "minimal_similarity" : 57.333332,
+      "maximum_similarity" : 71.666664,
+      "matched_token_number" : 43
+    },
+    "SortAlgo-SortAlgo4d2" : {
+      "minimal_similarity" : 68.75,
+      "maximum_similarity" : 78.57143,
+      "matched_token_number" : 44
+    },
+    "SortAlgo-SortAlgo4d1" : {
+      "minimal_similarity" : 69.84127,
+      "maximum_similarity" : 78.57143,
+      "matched_token_number" : 44
+    },
+    "SortAlgo1_4-SortAlgo3" : {
+      "minimal_similarity" : 34.782608,
+      "maximum_similarity" : 40.677967,
+      "matched_token_number" : 24
+    },
+    "SortAlgo3-SortAlgo3_5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_4-SortAlgo4" : {
+      "minimal_similarity" : 79.66102,
+      "maximum_similarity" : 79.66102,
+      "matched_token_number" : 47
+    },
+    "SortAlgo1_4-SortAlgo2" : {
+      "minimal_similarity" : 52.542374,
+      "maximum_similarity" : 55.357143,
+      "matched_token_number" : 31
+    },
+    "SortAlgo3-SortAlgo3_6" : {
+      "minimal_similarity" : 56.0,
+      "maximum_similarity" : 60.869564,
+      "matched_token_number" : 42
+    },
+    "SortAlgo1_4-SortAlgo7" : {
+      "minimal_similarity" : 52.542374,
+      "maximum_similarity" : 57.407406,
+      "matched_token_number" : 31
+    },
+    "SortAlgo1_4-SortAlgo5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo2-SortAlgo3_5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_4-SortAlgo6" : {
+      "minimal_similarity" : 40.298508,
+      "maximum_similarity" : 45.76271,
+      "matched_token_number" : 27
+    },
+    "SortAlgo2-SortAlgo3_6" : {
+      "minimal_similarity" : 32.0,
+      "maximum_similarity" : 42.857143,
+      "matched_token_number" : 24
+    },
+    "SortAlgo1-SortAlgo3_6" : {
+      "minimal_similarity" : 32.0,
+      "maximum_similarity" : 42.857143,
+      "matched_token_number" : 24
+    },
+    "SortAlgo1-SortAlgo3_5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo-SortAlgo1_2" : {
+      "minimal_similarity" : 59.210526,
+      "maximum_similarity" : 80.35714,
+      "matched_token_number" : 45
+    },
+    "SortAlgo1_3-SortAlgo3_5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_2-SortAlgo3_5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_2-SortAlgo3_6" : {
+      "minimal_similarity" : 31.578947,
+      "maximum_similarity" : 32.0,
+      "matched_token_number" : 24
+    },
+    "SortAlgo-SortAlgo1_6" : {
+      "minimal_similarity" : 56.666668,
+      "maximum_similarity" : 60.714287,
+      "matched_token_number" : 34
+    },
+    "SortAlgo-SortAlgo1_5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo-SortAlgo1_4" : {
+      "minimal_similarity" : 52.542374,
+      "maximum_similarity" : 55.357143,
+      "matched_token_number" : 31
+    },
+    "SortAlgo-SortAlgo1_3" : {
+      "minimal_similarity" : 53.333332,
+      "maximum_similarity" : 71.42857,
+      "matched_token_number" : 40
+    },
+    "SortAlgo1_6-SortAlgo4d1" : {
+      "minimal_similarity" : 36.50794,
+      "maximum_similarity" : 38.333332,
+      "matched_token_number" : 23
+    },
+    "SortAlgo4d2-SortAlgo7" : {
+      "minimal_similarity" : 62.5,
+      "maximum_similarity" : 74.07407,
+      "matched_token_number" : 40
+    },
+    "SortAlgo1_6-SortAlgo4d2" : {
+      "minimal_similarity" : 35.9375,
+      "maximum_similarity" : 38.333332,
+      "matched_token_number" : 23
+    },
+    "SortAlgo4d2-SortAlgo5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo4d2-SortAlgo6" : {
+      "minimal_similarity" : 34.328358,
+      "maximum_similarity" : 35.9375,
+      "matched_token_number" : 23
+    },
+    "SortAlgo1_6-SortAlgo4d3" : {
+      "minimal_similarity" : 37.096775,
+      "maximum_similarity" : 38.333332,
+      "matched_token_number" : 23
+    },
+    "SortAlgo1_5-SortAlgo2_5" : {
+      "minimal_similarity" : 100.0,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1_2-SortAlgo4d1" : {
+      "minimal_similarity" : 57.894737,
+      "maximum_similarity" : 69.84127,
+      "matched_token_number" : 44
+    },
+    "SortAlgo1_3-SortAlgo4d3" : {
+      "minimal_similarity" : 53.333332,
+      "maximum_similarity" : 64.51613,
+      "matched_token_number" : 40
+    },
+    "SortAlgo2-SortAlgo3" : {
+      "minimal_similarity" : 57.971016,
+      "maximum_similarity" : 71.42857,
+      "matched_token_number" : 40
+    },
+    "SortAlgo1_2-SortAlgo4d2" : {
+      "minimal_similarity" : 57.894737,
+      "maximum_similarity" : 68.75,
+      "matched_token_number" : 44
+    },
+    "SortAlgo1_3-SortAlgo4d2" : {
+      "minimal_similarity" : 53.333332,
+      "maximum_similarity" : 62.5,
+      "matched_token_number" : 40
+    },
+    "SortAlgo2-SortAlgo4" : {
+      "minimal_similarity" : 35.59322,
+      "maximum_similarity" : 37.5,
+      "matched_token_number" : 21
+    },
+    "SortAlgo1_6-SortAlgo2_5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_4-SortAlgo4d1" : {
+      "minimal_similarity" : 36.50794,
+      "maximum_similarity" : 38.98305,
+      "matched_token_number" : 23
+    },
+    "SortAlgo1_5-SortAlgo4d3" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_4-SortAlgo4d2" : {
+      "minimal_similarity" : 35.9375,
+      "maximum_similarity" : 38.98305,
+      "matched_token_number" : 23
+    },
+    "SortAlgo1_5-SortAlgo4d2" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_4-SortAlgo4d3" : {
+      "minimal_similarity" : 37.096775,
+      "maximum_similarity" : 38.98305,
+      "matched_token_number" : 23
+    },
+    "SortAlgo1_5-SortAlgo4d1" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo4d2-SortAlgo4d3" : {
+      "minimal_similarity" : 93.75,
+      "maximum_similarity" : 96.77419,
+      "matched_token_number" : 60
+    },
+    "SortAlgo2-SortAlgo7" : {
+      "minimal_similarity" : 82.14286,
+      "maximum_similarity" : 85.18519,
+      "matched_token_number" : 46
+    },
+    "SortAlgo1_2-SortAlgo4d3" : {
+      "minimal_similarity" : 57.894737,
+      "maximum_similarity" : 70.96774,
+      "matched_token_number" : 44
+    },
+    "SortAlgo1_3-SortAlgo4d1" : {
+      "minimal_similarity" : 53.333332,
+      "maximum_similarity" : 63.49206,
+      "matched_token_number" : 40
+    },
+    "SortAlgo2-SortAlgo5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo4d1-SortAlgo4d2" : {
+      "minimal_similarity" : 98.4375,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 63
+    },
+    "SortAlgo2-SortAlgo6" : {
+      "minimal_similarity" : 43.28358,
+      "maximum_similarity" : 51.785713,
+      "matched_token_number" : 29
+    },
+    "SortAlgo4d1-SortAlgo4d3" : {
+      "minimal_similarity" : 95.2381,
+      "maximum_similarity" : 96.77419,
+      "matched_token_number" : 60
+    },
+    "SortAlgo1_6-SortAlgo7" : {
+      "minimal_similarity" : 56.666668,
+      "maximum_similarity" : 62.962963,
+      "matched_token_number" : 34
+    },
+    "SortAlgo2_5-SortAlgo3_5" : {
+      "minimal_similarity" : 59.420288,
+      "maximum_similarity" : 73.21429,
+      "matched_token_number" : 41
+    },
+    "SortAlgo3_6-SortAlgo6" : {
+      "minimal_similarity" : 73.333336,
+      "maximum_similarity" : 82.089554,
+      "matched_token_number" : 55
+    },
+    "SortAlgo2_5-SortAlgo3_6" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo3_6-SortAlgo5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo3_6-SortAlgo7" : {
+      "minimal_similarity" : 32.0,
+      "maximum_similarity" : 44.444443,
+      "matched_token_number" : 24
+    },
+    "SortAlgo4d3-SortAlgo6" : {
+      "minimal_similarity" : 34.328358,
+      "maximum_similarity" : 37.096775,
+      "matched_token_number" : 23
+    },
+    "SortAlgo4d3-SortAlgo7" : {
+      "minimal_similarity" : 64.51613,
+      "maximum_similarity" : 74.07407,
+      "matched_token_number" : 40
+    },
+    "SortAlgo4d3-SortAlgo5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_4-SortAlgo2_5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1-SortAlgo4" : {
+      "minimal_similarity" : 35.59322,
+      "maximum_similarity" : 37.5,
+      "matched_token_number" : 21
+    },
+    "SortAlgo1-SortAlgo5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1-SortAlgo2" : {
+      "minimal_similarity" : 100.0,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo3_6-SortAlgo4" : {
+      "minimal_similarity" : 24.0,
+      "maximum_similarity" : 30.508474,
+      "matched_token_number" : 18
+    },
+    "SortAlgo1-SortAlgo3" : {
+      "minimal_similarity" : 57.971016,
+      "maximum_similarity" : 71.42857,
+      "matched_token_number" : 40
+    },
+    "SortAlgo1_3-SortAlgo2_5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_6-SortAlgo2" : {
+      "minimal_similarity" : 56.666668,
+      "maximum_similarity" : 60.714287,
+      "matched_token_number" : 34
+    },
+    "SortAlgo1-SortAlgo6" : {
+      "minimal_similarity" : 43.28358,
+      "maximum_similarity" : 51.785713,
+      "matched_token_number" : 29
+    },
+    "SortAlgo1-SortAlgo7" : {
+      "minimal_similarity" : 82.14286,
+      "maximum_similarity" : 85.18519,
+      "matched_token_number" : 46
+    },
+    "SortAlgo1_2-SortAlgo2_5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo-SortAlgo2_5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_6-SortAlgo6" : {
+      "minimal_similarity" : 82.089554,
+      "maximum_similarity" : 91.666664,
+      "matched_token_number" : 55
+    },
+    "SortAlgo1_6-SortAlgo5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_6-SortAlgo4" : {
+      "minimal_similarity" : 35.0,
+      "maximum_similarity" : 35.59322,
+      "matched_token_number" : 21
+    },
+    "SortAlgo1_6-SortAlgo3" : {
+      "minimal_similarity" : 34.782608,
+      "maximum_similarity" : 40.0,
+      "matched_token_number" : 24
+    },
+    "SortAlgo1_5-SortAlgo6" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_5-SortAlgo7" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo2_5-SortAlgo7" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1-SortAlgo4d3" : {
+      "minimal_similarity" : 70.96774,
+      "maximum_similarity" : 78.57143,
+      "matched_token_number" : 44
+    },
+    "SortAlgo1_5-SortAlgo4" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1-SortAlgo4d2" : {
+      "minimal_similarity" : 68.75,
+      "maximum_similarity" : 78.57143,
+      "matched_token_number" : 44
+    },
+    "SortAlgo1_5-SortAlgo5" : {
+      "minimal_similarity" : 100.0,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1-SortAlgo4d1" : {
+      "minimal_similarity" : 69.84127,
+      "maximum_similarity" : 78.57143,
+      "matched_token_number" : 44
+    },
+    "SortAlgo1_5-SortAlgo2" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo3_5-SortAlgo6" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_5-SortAlgo3" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo3_5-SortAlgo7" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo3-SortAlgo4d1" : {
+      "minimal_similarity" : 56.52174,
+      "maximum_similarity" : 61.904762,
+      "matched_token_number" : 39
+    },
+    "SortAlgo3-SortAlgo4d3" : {
+      "minimal_similarity" : 56.52174,
+      "maximum_similarity" : 62.903225,
+      "matched_token_number" : 39
+    },
+    "SortAlgo3-SortAlgo4d2" : {
+      "minimal_similarity" : 56.52174,
+      "maximum_similarity" : 60.9375,
+      "matched_token_number" : 39
+    },
+    "SortAlgo2-SortAlgo4d3" : {
+      "minimal_similarity" : 70.96774,
+      "maximum_similarity" : 78.57143,
+      "matched_token_number" : 44
+    },
+    "SortAlgo2-SortAlgo4d1" : {
+      "minimal_similarity" : 69.84127,
+      "maximum_similarity" : 78.57143,
+      "matched_token_number" : 44
+    },
+    "SortAlgo1_5-SortAlgo1_6" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo2-SortAlgo4d2" : {
+      "minimal_similarity" : 68.75,
+      "maximum_similarity" : 78.57143,
+      "matched_token_number" : 44
+    },
+    "SortAlgo3_6-SortAlgo4d2" : {
+      "minimal_similarity" : 30.666666,
+      "maximum_similarity" : 35.9375,
+      "matched_token_number" : 23
+    },
+    "SortAlgo4d1-SortAlgo7" : {
+      "minimal_similarity" : 63.49206,
+      "maximum_similarity" : 74.07407,
+      "matched_token_number" : 40
+    },
+    "SortAlgo3_6-SortAlgo4d1" : {
+      "minimal_similarity" : 30.666666,
+      "maximum_similarity" : 36.50794,
+      "matched_token_number" : 23
+    },
+    "SortAlgo4d1-SortAlgo6" : {
+      "minimal_similarity" : 34.328358,
+      "maximum_similarity" : 36.50794,
+      "matched_token_number" : 23
+    },
+    "SortAlgo4-SortAlgo4d1" : {
+      "minimal_similarity" : 28.571428,
+      "maximum_similarity" : 30.508474,
+      "matched_token_number" : 18
+    },
+    "SortAlgo3_6-SortAlgo4d3" : {
+      "minimal_similarity" : 30.666666,
+      "maximum_similarity" : 37.096775,
+      "matched_token_number" : 23
+    },
+    "SortAlgo4-SortAlgo4d2" : {
+      "minimal_similarity" : 28.125,
+      "maximum_similarity" : 30.508474,
+      "matched_token_number" : 18
+    },
+    "SortAlgo4-SortAlgo4d3" : {
+      "minimal_similarity" : 29.032259,
+      "maximum_similarity" : 30.508474,
+      "matched_token_number" : 18
+    },
+    "SortAlgo3_5-SortAlgo4" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo3_5-SortAlgo4d2" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo4d1-SortAlgo5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo3_5-SortAlgo4d3" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo3_5-SortAlgo5" : {
+      "minimal_similarity" : 59.420288,
+      "maximum_similarity" : 73.21429,
+      "matched_token_number" : 41
+    },
+    "SortAlgo2_5-SortAlgo4" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo2_5-SortAlgo3" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo3_5-SortAlgo4d1" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo2_5-SortAlgo6" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo2_5-SortAlgo5" : {
+      "minimal_similarity" : 100.0,
+      "maximum_similarity" : 100.0,
+      "matched_token_number" : 56
+    },
+    "SortAlgo1-SortAlgo1_3" : {
+      "minimal_similarity" : 53.333332,
+      "maximum_similarity" : 71.42857,
+      "matched_token_number" : 40
+    },
+    "SortAlgo1-SortAlgo1_2" : {
+      "minimal_similarity" : 59.210526,
+      "maximum_similarity" : 80.35714,
+      "matched_token_number" : 45
+    },
+    "SortAlgo1-SortAlgo1_6" : {
+      "minimal_similarity" : 56.666668,
+      "maximum_similarity" : 60.714287,
+      "matched_token_number" : 34
+    },
+    "SortAlgo1-SortAlgo1_5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1-SortAlgo1_4" : {
+      "minimal_similarity" : 52.542374,
+      "maximum_similarity" : 55.357143,
+      "matched_token_number" : 31
+    },
+    "SortAlgo2_5-SortAlgo4d1" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo3-SortAlgo7" : {
+      "minimal_similarity" : 57.971016,
+      "maximum_similarity" : 74.07407,
+      "matched_token_number" : 40
+    },
+    "SortAlgo2_5-SortAlgo4d2" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo3-SortAlgo6" : {
+      "minimal_similarity" : 52.173912,
+      "maximum_similarity" : 53.731342,
+      "matched_token_number" : 36
+    },
+    "SortAlgo1_4-SortAlgo1_5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo3-SortAlgo5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_4-SortAlgo1_6" : {
+      "minimal_similarity" : 51.666668,
+      "maximum_similarity" : 52.542374,
+      "matched_token_number" : 31
+    },
+    "SortAlgo3-SortAlgo4" : {
+      "minimal_similarity" : 26.086956,
+      "maximum_similarity" : 30.508474,
+      "matched_token_number" : 18
+    },
+    "SortAlgo1_3-SortAlgo1_5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_3-SortAlgo1_4" : {
+      "minimal_similarity" : 32.0,
+      "maximum_similarity" : 40.677967,
+      "matched_token_number" : 24
+    },
+    "SortAlgo2_5-SortAlgo4d3" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_3-SortAlgo1_6" : {
+      "minimal_similarity" : 32.0,
+      "maximum_similarity" : 40.0,
+      "matched_token_number" : 24
+    },
+    "SortAlgo-SortAlgo3_6" : {
+      "minimal_similarity" : 32.0,
+      "maximum_similarity" : 42.857143,
+      "matched_token_number" : 24
+    },
+    "SortAlgo1_2-SortAlgo1_5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo-SortAlgo3_5" : {
+      "minimal_similarity" : 0.0,
+      "maximum_similarity" : 0.0,
+      "matched_token_number" : 0
+    },
+    "SortAlgo1_2-SortAlgo1_6" : {
+      "minimal_similarity" : 44.736843,
+      "maximum_similarity" : 56.666668,
+      "matched_token_number" : 34
+    },
+    "SortAlgo1_2-SortAlgo1_3" : {
+      "minimal_similarity" : 52.63158,
+      "maximum_similarity" : 53.333332,
+      "matched_token_number" : 40
+    },
+    "SortAlgo1_2-SortAlgo1_4" : {
+      "minimal_similarity" : 40.789474,
+      "maximum_similarity" : 52.542374,
+      "matched_token_number" : 31
+    }
+  }
+} ]