Class ProjectFiles


  • public final class ProjectFiles
    extends java.lang.Object
    Manages all files for one Patterns Learn project. This class: 1. Knows and maintains a specific folder and file naming structure that the Learn application uses to store projects. 2. Contains factory methods that construct ProjectSettings and ModelSettings objects while creating the necessary files and folders. 3. Provides methods that modify both project and model XML files. 4. Ensures consistency between the name of project/model stored in XML file and the project/model directory name.
    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      DataPartition addRecPair​(RecPair recPair)
      Adds a new record pair randomly to the training or validation dataset of the active model.
      void addRecPair​(RecPair recPair, DataPartition partition)
      Adds a new record pair to the specified dataset of the active model.
      void addRecPairsFromCsv​(java.io.File csvFile)
      Adds all record pairs from the specified CSV file to the active model settings.
      void addThesFile​(java.lang.String filePath)
      Adds the thesaurus file to the "thesauri subdirectory of the active model's directory, overwriting any previous file with same name.
      AddPairsResult addUsefulRecPairs​(java.util.List<RecPair> recPairs, int maxTrainSize, int maxReserveSize)
      Adds each record pair if it is likely to improve model training while limiting the grwth of datasets.
      static ProjectFiles create​(java.lang.String projectName, java.lang.String projectPath)
      Creates a new project.
      LowConfPairFinder createLowConfPairFinder()
      Creates and stores an object that finds low confidence pairs in the data table.
      ModelTrainConfig createModelTrainConfig​(ModelConfig mConfig, java.lang.Boolean randomOrder, java.util.Map<Partition,​ConvergenceObserver> observers)
      Creates and remembers a new ModelTrainConfig, replacing any previous one.
      RecPairScorer createScorer​(com.netrics.likeit.NetricsServerInterface server)
      Creates a RecPairScorer that can be used only for scoring any record pairs with known field values.
      ModelTrainer createTrainer​(com.netrics.likeit.NetricsServerInterface server)
      Creates a ModelTrainer for training the active model.
      void deleteModel​(int index)
      Deletes the non-active model specified by index.
      void deleteTrainedModel()
      Deletes binary file of the trained model.
      void exportModelAndQuery​(int modelIndex, java.nio.file.Path exportedModelFile, java.lang.String modelVersion, java.nio.file.Path queryOutputDir, java.lang.String javaClassName, java.lang.String methodName)
      Exports all files needed to use the trained model on the Patterns server: the trained model binary file, Java source file that builds the Learn query, and any thesaurus files used by the query.
      void exportThesFiles​(java.nio.file.Path outputDir)
      Exports all thesaurus files used by the feature query of the active model.
      void exportTrainedModel​(java.nio.file.Path exportedFile)
      Exports the trained model binary file with model score threshold.
      void exportTrainedModel​(java.nio.file.Path exportedFile, java.lang.String version)
      Exports the trained model binary file from the active ModelSettings to the given location.
      ModelSettings getActiveModelSettings()  
      java.nio.file.Path getDataFile​(int fileIndex)  
      ModelTrainConfig getModelTrainConfig()
      Get model training configuration.
      int getNModels()  
      ProjectSettings getProjectSettings()  
      boolean hasLowConfPairFinder()  
      boolean hasModelTrainConfig()
      Checks whether a custom model training configuration is used to train the model.
      static ProjectFiles load​(java.lang.String projectName, java.lang.String projectPath)
      Loads an existing project from the specified directory.
      ModelSettings loadModel​(int index)
      Loads ModelSettings from model.xml file specified by the index.
      void loadTrainedModel()
      Loads the trained model from the model.bin file to ModelTrainer of the active model.
      boolean localDataFileExists​(java.lang.String filePath)  
      AddPairsResult moveUsefulRecPairs​(DataPartition sourcePartition, int maxTrainSize)
      Moves the record pairs that are likely to improve model training from the given dataset (Reserve) to the training or validation dataset of the active model.
      void replaceDataFile​(int fileIndex, java.lang.String filePath, java.util.List<java.lang.String> fieldNames, boolean copyFile, boolean overwriteFile)
      Replaces the currently assigned data file with a compatible data file.
      void save()
      Saves project.xml and active model.xml files to the existing project location
      void saveAs​(java.lang.String projectName, java.lang.String projectPath)
      Saves project files to the new project location and/or project name.
      void saveLowConfPairFinder​(int maxNPairs)
      Saves the state of the low confidence pair finder that was created before and used to find pairs.
      void saveModelCopy​(java.lang.String copiedModelName)
      Saves the active model, then saves its copy (creates model dir, model.xml and model.bin).
      void saveTrainedModel()
      Saves trained model binary file to the active model directory.
      int setDataFile​(java.lang.String filePath, java.util.List<java.lang.String> fieldNames, boolean copyFile, boolean overwriteFile)
      Sets the data file info.
      void setDebugOutput​(boolean value)
      Sets the debug output.
      void setKeyField​(int fieldIndex)
      Sets the index of the key field in the data file.
      ModelSettings switchActiveModel​(int index)
      Saves the currently active model, then loads another model from file, makes it active, saves the new active model index.
      boolean thesFileExists​(java.lang.String filePath)  
      java.lang.String toString()  
      void updateDataFileLastModified​(int fileIndex)
      Updates the stored last modified date of the data file.
      void verifyCanSetDataFile()
      Verifies that a data file can be assigned or replaced.
      static void verifyDir​(java.lang.String projectName, java.lang.String projectPath)
      Verifies that the structure of files and directories of the given project is correct without loading the entire project.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
    • Method Detail

      • create

        public static ProjectFiles create​(java.lang.String projectName,
                                          java.lang.String projectPath)
                                   throws java.io.IOException,
                                          org.xml.sax.SAXException,
                                          javax.xml.bind.JAXBException
        Creates a new project. Creates project directory structure, saves the project settings and current model settings. Uses random partitioning for pairs.
        Parameters:
        projectName - - the name of the new project directory. Must not exist. Not null.
        projectPath - - path where the project directory will be created. The path must exist. Not null.
        Returns:
        newly constructed ProjectFiles object.
        Throws:
        org.xml.sax.SAXException - if the schema file cannot be parsed.
        javax.xml.bind.JAXBException - when XML validation fails.
        java.nio.file.FileAlreadyExistsException - if project directory exists (optional).
        java.io.IOException - if project directory cannot be created.
      • load

        public static ProjectFiles load​(java.lang.String projectName,
                                        java.lang.String projectPath)
                                 throws org.xml.sax.SAXException,
                                        javax.xml.bind.JAXBException,
                                        java.io.FileNotFoundException
        Loads an existing project from the specified directory. Uses random partitioning for pairs.
        Parameters:
        projectName - - the name of the project directory (must exist). Not null.
        projectPath - - path to the project directory. Not null.
        Returns:
        newly constructed ProjectFiles object with data loaded from files.
        Throws:
        javax.xml.bind.JAXBException - if an XML file validation fails, or if an XML file was created with newer Learn API.
        org.xml.sax.SAXException - if the schema file cannot be parsed.
        java.io.FileNotFoundException - if a required project file or directory is not found.
        java.lang.IndexOutOfBoundsException - if active model index is invalid.
      • verifyDir

        public static void verifyDir​(java.lang.String projectName,
                                     java.lang.String projectPath)
                              throws java.io.FileNotFoundException
        Verifies that the structure of files and directories of the given project is correct without loading the entire project. This only verifies the presence of files and directories that are required by any Learn project. It does not verify the content of each project file (loading the project is required to do that).
        Parameters:
        projectName - - the name of the project directory. Not null.
        projectPath - - path to the project directory. Not null.
        Throws:
        java.io.FileNotFoundException - if a required project directory or file does not exist.
      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object
      • getProjectSettings

        public ProjectSettings getProjectSettings()
        Returns:
        internal reference to the ProjectSettings object.
      • setDebugOutput

        public void setDebugOutput​(boolean value)
        Sets the debug output. Used for LowConfPairFinder.
        Parameters:
        value - - if true, debug information is outputted to console.
      • save

        public void save()
                  throws javax.xml.bind.JAXBException
        Saves project.xml and active model.xml files to the existing project location
        Throws:
        javax.xml.bind.JAXBException - when XML validation fails.
      • saveAs

        public void saveAs​(java.lang.String projectName,
                           java.lang.String projectPath)
                    throws javax.xml.bind.JAXBException,
                           java.io.IOException
        Saves project files to the new project location and/or project name. New project directory will be created (must not exist). This ProjectFiles object refers to the new location after the call. The files in the old project location are not modified. Note: any data file is NOT copied (currently client code manages data file location).
        Parameters:
        projectName - - name of the project directory. Not null.
        projectPath - - path to the project directory. Not null.
        Throws:
        javax.xml.bind.JAXBException - when XML validation fails.
        java.nio.file.FileAlreadyExistsException - if project directory exists (optional).
        java.io.IOException - if directory cannot be created or other I/O error occurs.
      • getActiveModelSettings

        public ModelSettings getActiveModelSettings()
        Returns:
        internal reference to the ModelSettings object for the active model that was selected by the user. By default the current (most recent) model is active.
      • getNModels

        public int getNModels()
        Returns:
        the number of saved models (model.xml files, ModelSettings objects).
      • loadModel

        public ModelSettings loadModel​(int index)
                                throws javax.xml.bind.JAXBException,
                                       java.io.FileNotFoundException
        Loads ModelSettings from model.xml file specified by the index. Can be used to read info for any model. Any changes to the returned object cannot be saved (the returned object is not retained by ProjectFiles, and ModelSettings does not have any public save methods. Use getActiveModelSettings() to make and save changes).
        Parameters:
        index - - index of the model in the list of models (in ProjectSettings).
        Returns:
        newly created ModelSettings object.
        Throws:
        javax.xml.bind.JAXBException - if model XML file validation fails or if the XML file was created with newer Learn API.
        java.io.FileNotFoundException - if the model XML file is not found.
        java.lang.IndexOutOfBoundsException - if model index is invalid.
      • switchActiveModel

        public ModelSettings switchActiveModel​(int index)
                                        throws javax.xml.bind.JAXBException,
                                               java.io.FileNotFoundException
        Saves the currently active model, then loads another model from file, makes it active, saves the new active model index. Any previously obtained references to the active ModelSettings object and its derived objects (FeatureQuery, ModelTrainer) become obsolete and should no longer be used.
        Parameters:
        index - - index of the model in the list of models (in ProjectSettings).
        Returns:
        ModelSettings object for the new active model.
        Throws:
        javax.xml.bind.JAXBException - if an XML file validation fails, or if an XML file was created with newer Learn API.
        java.io.FileNotFoundException - if the model XML file is not found.
      • saveModelCopy

        public void saveModelCopy​(java.lang.String copiedModelName)
                           throws javax.xml.bind.JAXBException,
                                  java.io.IOException
        Saves the active model, then saves its copy (creates model dir, model.xml and model.bin). Adds the copied model to the list of models in ProjectSettings. After the call, the active model is still the same model.xml, not the new copy.
        Parameters:
        copiedModelName - - the name of the new copy of the active model. Must be different from the name of any other saved model.
        Throws:
        javax.xml.bind.JAXBException - when XML validation fails.
        java.nio.file.FileAlreadyExistsException - if the new model directory exists (optional).
        java.io.IOException - if the new model directory cannot be created.
        java.nio.file.InvalidPathException - if copiedModelName is not a valid directory name (optional).
        java.lang.IllegalArgumentException - if copiedModelName is null, empty, or 'current'
      • deleteModel

        public void deleteModel​(int index)
                         throws java.io.IOException,
                                javax.xml.bind.JAXBException
        Deletes the non-active model specified by index. Deletes model files and directory, saves the updated project. The index of the active model may change. Note that deleting a model may break the continuity of the tree structure of saved models.
        Parameters:
        index - - index of the model in the list of models (in ProjectSettings).
        Throws:
        java.lang.IllegalArgumentException - if the model to be deleted is currently active.
        javax.xml.bind.JAXBException - when XML validation fails when saving project. Model files were not deleted.
        java.io.IOException - if one of the model files or directory could not be deleted. In this case the model list no longer contains the deleted model, but model files may still exist.
      • localDataFileExists

        public boolean localDataFileExists​(java.lang.String filePath)
        Returns:
        true if the file with the specified name exists in project's "tables" directory.
      • setDataFile

        public int setDataFile​(java.lang.String filePath,
                               java.util.List<java.lang.String> fieldNames,
                               boolean copyFile,
                               boolean overwriteFile)
                        throws java.io.IOException
        Sets the data file info. Assumes only 1 data file can be used. Fills in the fieldDefs in both project and active model XML files based on given fieldNames taken from the data file. Assigns field type Searchable Text to all fields. Removes key field index, feature query, pair finder settings. Must be followed by setKeyField(int) that creates feature query.
        Parameters:
        filePath - - the data file including the absolute path.
        fieldNames - defines all field names and field order. Indexes of these field names are later used to identify a specific field.
        copyFile - - if true, copies the data file to the project directory.
        overwriteFile - - overwrite any existing local data file while copying.
        Returns:
        index of the added file (currently always 0).
        Throws:
        java.io.FileNotFoundException - if the specified data file is not found (optional).
        java.nio.file.FileAlreadyExistsException - if local file exists during copying and overwriteFile==false (optional).
        java.io.IOException - if file I/O error occurs during copying or other operation.
        java.lang.IllegalArgumentException - - if fieldNames is not given or has empty or duplicate names
        java.lang.IllegalStateException - - if existing data file cannot be replaced: features, record pairs or trained model already assigned, or several models have been saved for the same project.
      • replaceDataFile

        public void replaceDataFile​(int fileIndex,
                                    java.lang.String filePath,
                                    java.util.List<java.lang.String> fieldNames,
                                    boolean copyFile,
                                    boolean overwriteFile)
                             throws java.io.IOException
        Replaces the currently assigned data file with a compatible data file. The new data file must have the same field names, types, and same key field. Replacement can be performed even when features, pairs or trained models exist. Does not change already assigned fields (types, statistics, key field, ignored fields). Removes pair finder settings. Field statistics should be reassigned after replacing the data file.
        Parameters:
        fileIndex - - index of the data file: from 0 to ProjectSettings.getNDataFiles()-1
        filePath - - the data file including the absolute path.
        fieldNames - - field names read from the new data file. Must match old field names.
        copyFile - - if true, copies the data file to the project directory.
        overwriteFile - - overwrite any existing local data file while copying.
        Throws:
        java.io.FileNotFoundException - if the specified data file is not found (optional).
        java.nio.file.FileAlreadyExistsException - if local file exists during copying and overwriteFile==false (optional).
        java.io.IOException - if file I/O error occurs during copying or other operation.
        java.lang.IndexOutOfBoundsException - if there is no data file with given index.
        java.lang.IllegalArgumentException - if fieldNames do not match already assigned field names.
      • updateDataFileLastModified

        public void updateDataFileLastModified​(int fileIndex)
                                        throws java.io.IOException
        Updates the stored last modified date of the data file. Determines the current date of the existing data file and writes it to XML. Removes pair finder settings. Use with caution: existing pairs must match records in the data file, otherwise operations that use the server table may fail.
        Parameters:
        fileIndex - - index of the data file: from 0 to ProjectSettings.getNDataFiles()-1
        Throws:
        java.io.IOException - if I/O error occurs while obtaining file date
        java.lang.IndexOutOfBoundsException - if there is no data file with given index.
      • verifyCanSetDataFile

        public void verifyCanSetDataFile()
        Verifies that a data file can be assigned or replaced.
        Throws:
        java.lang.IllegalStateException - - if existing data file cannot be replaced (features, record pairs or trained model already assigned). Also if several models have been saved for the same project.
      • getDataFile

        public java.nio.file.Path getDataFile​(int fileIndex)
        Parameters:
        fileIndex - - index of the data file: from 0 to ProjectSettings.getNDataFiles()-1
        Returns:
        object containing the absolute path to the data file.
        Throws:
        java.lang.IndexOutOfBoundsException - if there is no data file with given index.
      • setKeyField

        public void setKeyField​(int fieldIndex)
        Sets the index of the key field in the data file. Creates empty feature query.
        Parameters:
        fieldIndex - - the index of key field in the list of field names of the data file.
        Throws:
        java.lang.IllegalStateException - if data file is not assigned; or several models have been saved; or active model has features/pairs assigned.
        java.lang.IllegalArgumentException - if the given field is ignored.
      • thesFileExists

        public boolean thesFileExists​(java.lang.String filePath)
        Returns:
        true if the file with the specified name exists in active model's "thesauri" directory.
      • addThesFile

        public void addThesFile​(java.lang.String filePath)
                         throws java.io.IOException
        Adds the thesaurus file to the "thesauri subdirectory of the active model's directory, overwriting any previous file with same name. Overwriting will cause all features that use old thesaurus file to use the new file. Check it first using thesFileExists(String).
        Parameters:
        filePath - - the thesaurus file including the absolute path.
        Throws:
        java.io.IOException - if file copying fails.
      • exportThesFiles

        public void exportThesFiles​(java.nio.file.Path outputDir)
                             throws java.io.IOException
        Exports all thesaurus files used by the feature query of the active model. If a file with the same name exists in output directory, this method overwrites it.
        Parameters:
        outputDir - - the directory where thesaurus files are copied to.
        Throws:
        java.io.IOException - if I/O error occurs
      • addRecPair

        public DataPartition addRecPair​(RecPair recPair)
        Adds a new record pair randomly to the training or validation dataset of the active model. Saves info about the currently assigned data file as the source of the pair. Data file must be assigned and contain the given records.
        Parameters:
        recPair - - information on the two records. Not null.
        Returns:
        the partition to which the record pair was added (training or validation)
        Throws:
        java.lang.IllegalStateException - if data file or key field is not assigned.
        java.lang.IllegalArgumentException - if rec pair with same keys has already been added to one of the datasets, or if the record pair contains incorrect number of fields or feature values.
      • addRecPair

        public void addRecPair​(RecPair recPair,
                               DataPartition partition)
        Adds a new record pair to the specified dataset of the active model. Saves info about the currently assigned data file as the source of the pair. Data file must be assigned and contain the given records. To add a pair to the training or validation dataset addRecPair(RecPair) must be used.
        Parameters:
        recPair - - information on the two records. Not null.
        partition - - the dataset to add the pair to. It must not be used in training.
        Throws:
        java.lang.IllegalStateException - if data file or key field is not assigned.
        java.lang.IllegalArgumentException - if rec pair with same keys has already been added to one of the datasets, or if the record pair contains incorrect number of fields or feature values, or if partition is used in training.
      • addUsefulRecPairs

        public AddPairsResult addUsefulRecPairs​(java.util.List<RecPair> recPairs,
                                                int maxTrainSize,
                                                int maxReserveSize)
                                         throws com.netrics.likeit.NetricsException,
                                                java.io.IOException
        Adds each record pair if it is likely to improve model training while limiting the grwth of datasets. Useful pairs are added randomly to the training or validation dataset of the active model. Before calling this method, create model trainer with table definition and load trained model if it exists. If trained model does not exist, the growth of datasets is potentially unlimited.
        Parameters:
        recPairs - - all record pairs to be added.
        maxTrainSize - - no more pairs will be added to Train/Vld datasets if the Training dataset reaches this maximum size. Should be as high as possible (3000 and more) as long as the model training time is acceptable.
        maxReserveSize - - the maximum number of pairs that can be stored to the Reserve dataset. Pairs that are deemed not useful are added to the Reserve dataset up to this limit. If 0, no pairs are added. If -1, an unlimited number of pairs are added.
        Returns:
        an object containing groups of pairs added and not added to project.
        Throws:
        java.lang.IllegalStateException - if data file or key field is not assigned, or if model trainer has not been created, or if neither the server table nor table definition is assigned, or if the server table is assigned but not loaded, or if trained model exists but was not loaded to model trainer, or if the feature query is empty.
        java.lang.IllegalArgumentException - if any record pair contains incorrect number of fields or feature values.
        com.netrics.likeit.NetricsException - if the server indicates that an error has occured
        java.io.IOException - if an I/O error occurs while communicating with server
      • moveUsefulRecPairs

        public AddPairsResult moveUsefulRecPairs​(DataPartition sourcePartition,
                                                 int maxTrainSize)
                                          throws com.netrics.likeit.NetricsException,
                                                 java.io.IOException
        Moves the record pairs that are likely to improve model training from the given dataset (Reserve) to the training or validation dataset of the active model. Before calling this, create model trainer with table definition and load trained model if it exists.
        Parameters:
        sourcePartition - - the dataset with the pairs to be considered. Must be a dataset not used for training (currently only the RESERVE dataset).
        maxTrainSize - - no more pairs will be added to Train/Vld datasets if the Training dataset reaches this maximum size. Should be as high as possible (3000 and more) as long as the model training time is acceptable.
        Returns:
        an object containing groups of pairs added to Training and Validation datasets (remaining lists of pairs are not used).
        Throws:
        java.lang.IllegalStateException - if data file or key field is not assigned, or if model trainer has not been created, or if neither the server table nor table definition is assigned, or if the server table is assigned but not loaded, or if trained model exists but was not loaded to model trainer, or if the feature query is empty.
        java.lang.IllegalArgumentException - if sourcePartition is not RESERVE.
        com.netrics.likeit.NetricsException - if the server indicates that an error has occured
        java.io.IOException - if an I/O error occurs while communicating with server
      • addRecPairsFromCsv

        public void addRecPairsFromCsv​(java.io.File csvFile)
                                throws com.netrics.likeit.NetricsFileFormatException,
                                       java.io.IOException,
                                       com.netrics.likeit.NetricsException
        Adds all record pairs from the specified CSV file to the active model settings. Format of each line in CSV file:"key1,key2,label", cannot contain spaces before/after comma. The keys must exist in the assigned data file. Field values are read from the server table. Label is "0" (False), "1" (True), "-" (null), "?" or any other value (Unsure).
        Parameters:
        csvFile - - the CSV file with record pairs
        Throws:
        java.lang.IllegalStateException - if data file or key field is not assigned; or the ModelTrainer is not created or does not have a server table assigned.
        java.io.FileNotFoundException - if the CSV file is not found (optional).
        com.netrics.likeit.NetricsFileFormatException - if CSV file format is incorrect
        com.netrics.likeit.NetricsException - if the server indicates that an error has occured
        java.io.IOException - if pairs CSV file has wrong number of entries per line; or I/O error occurs while communicating with server
      • createTrainer

        public ModelTrainer createTrainer​(com.netrics.likeit.NetricsServerInterface server)
                                   throws com.netrics.likeit.NetricsException,
                                          java.io.IOException
        Creates a ModelTrainer for training the active model. Closes any existing trainer. Note: client must still call ModelTrainer.close(boolean) after scorer or RLink model has been created and the ModelTrainer object will not be replaced by calling this method.
        Parameters:
        server - - Patterns server must be running and accessible.
        Throws:
        com.netrics.likeit.NetricsException - if an error occured on server.
        java.io.IOException - if the server does not exist, is not accessible or if an I/O error occurs while communicating with server.
      • createScorer

        public RecPairScorer createScorer​(com.netrics.likeit.NetricsServerInterface server)
                                   throws com.netrics.likeit.NetricsException,
                                          java.io.IOException
        Creates a RecPairScorer that can be used only for scoring any record pairs with known field values. Client must close the returned scorer object when it is no longer used. Scorer will use the field names, types and feature query defined for the active model. This scorer object does not load or require a table on server. Use ModelTrainer to score pairs in the project, train models, etc. See createTrainer(NetricsServerInterface).
        Parameters:
        server - - Patterns server must be running and accessible.
        Returns:
        the newly created RecPairScorer.
        Throws:
        java.lang.IllegalStateException - if data file or key field is not assigned, or if no features are defined.
        com.netrics.likeit.NetricsException - if an error occured on server.
        java.io.IOException - if the server does not exist, is not accessible or if an I/O error occurs while communicating with server.
      • saveTrainedModel

        public void saveTrainedModel()
                              throws javax.xml.bind.JAXBException,
                                     com.netrics.likeit.NetricsException,
                                     java.io.IOException
        Saves trained model binary file to the active model directory. Stores in active model settings: binary file name, training date (current timestamp), training iteration, model score for each pair. Saves project/model XML files to disk (to be in sync with the saved binary file).
        Throws:
        java.lang.IllegalStateException - if ModelTrainer does not exist, or if no training iterations were performed.
        javax.xml.bind.JAXBException - - if XML validation fails or other JAXB error occurs.
        com.netrics.likeit.NetricsException - if the server error occured when deleting old loaded model.
        java.io.IOException - if the model file cannot be created.
      • hasModelTrainConfig

        public boolean hasModelTrainConfig()
        Checks whether a custom model training configuration is used to train the model.
        Returns:
        true if the modelTrainConfig.xml file exists in the project.
      • getModelTrainConfig

        public ModelTrainConfig getModelTrainConfig()
        Get model training configuration. Changes to the returned object will be visible in ProjectFiles and ModelTrainer (if it exists).
        Returns:
        the custom model training configuration (internal reference), or null if the project does not contain the modelTrainConfig.xml file.
      • createModelTrainConfig

        public ModelTrainConfig createModelTrainConfig​(ModelConfig mConfig,
                                                       java.lang.Boolean randomOrder,
                                                       java.util.Map<Partition,​ConvergenceObserver> observers)
                                                throws org.xml.sax.SAXException
        Creates and remembers a new ModelTrainConfig, replacing any previous one. After this call, modelTrainConfig.xml will be saved to project directory when the project is saved. Updates ModelTrainer to use the new ModelTrainConfig.
        Parameters:
        mConfig - - values to initialize the model configuration in ModelTrainConfig. If null, modelConfig information will be blank (default).
        randomOrder - - value for random order of training examples. Default if null.
        observers - - convergence observers to initialize training configuration in ModelTrainConfig. Must contain COErrorRateMin for Validation dataset and COErrorRate for Training dataset.
        Returns:
        the new ModelTrainConfig object with default settings.
        Throws:
        org.xml.sax.SAXException - if the schema file cannot be parsed.
        java.lang.IllegalArgumentException - if observers contain objects other than COErrorRate or COErrorRateMin.
      • deleteTrainedModel

        public void deleteTrainedModel()
                                throws javax.xml.bind.JAXBException,
                                       java.io.IOException
        Deletes binary file of the trained model. Deletes training statistics, predictions for all pairs, pair finder state from active ModelSettings. Saves project.
        Throws:
        java.lang.IllegalStateException - if trained model has not been saved.
        javax.xml.bind.JAXBException - - if XML validation fails.
        java.io.IOException - - if deleting the binary file fails.
      • loadTrainedModel

        public void loadTrainedModel()
                              throws java.io.FileNotFoundException
        Loads the trained model from the model.bin file to ModelTrainer of the active model.
        Throws:
        java.io.FileNotFoundException - if the model file is missing.
        java.lang.IllegalStateException - if ModelTrainer has not been created; or trained model was not saved in ModelSettings; or if a model is already created or loaded to ModelTrainer.
      • createLowConfPairFinder

        public LowConfPairFinder createLowConfPairFinder()
                                                  throws com.netrics.likeit.NetricsException,
                                                         java.io.IOException
        Creates and stores an object that finds low confidence pairs in the data table. If needed, creates scorer, loads the trained model to server and to the ModelTrainer. Any pairs saved from the last LowConfPairFinder are removed from the active model, and those that still have low confidence are stored in the returned object.
        Returns:
        the new pair finder object.
        Throws:
        java.lang.IllegalStateException - if trained model has not been saved, or if ModelTrainer has not been created, or if server table has not been assigned/loaded or if the table is empty, or if feature confidence measure is not used.
        java.io.FileNotFoundException - if the model file does not exist.
        com.netrics.likeit.NetricsException - if the server indicates that an error has occured.
        java.io.IOException - if an I/O error occurs while communicating with server.
      • hasLowConfPairFinder

        public boolean hasLowConfPairFinder()
        Returns:
        true of the pair finder has been created.
      • saveLowConfPairFinder

        public void saveLowConfPairFinder​(int maxNPairs)
        Saves the state of the low confidence pair finder that was created before and used to find pairs. Saves the table position and confidence threshold in the active model. The found unprocessed pairs are saved in the LOW_CONF dataset. Any previous pairs in that dataset are deleted. Unprocessed pairs are not saved if they already exist in the project or are very similar to the last returned pair. Thus the LOW_CONF dataset will empty if all the unprocessed pairs were skipped.
        Parameters:
        maxNPairs - - maximum number of found pairs to save.
        Throws:
        java.lang.IllegalStateException - if a pair finder has not been created.
      • exportTrainedModel

        public void exportTrainedModel​(java.nio.file.Path exportedFile,
                                       java.lang.String version)
                                throws java.io.IOException
        Exports the trained model binary file from the active ModelSettings to the given location. Saves model score threshold and the given version in the exported file.
        Parameters:
        exportedFile - - destination binary model file.
        version - - new version value for model file. If null, version is not assigned.
        Throws:
        java.nio.file.FileAlreadyExistsException - if the given file already exists (optional).
        java.io.IOException - for various file copy errors
        java.lang.IllegalStateException - if trained model has not been saved for the active model.
      • exportTrainedModel

        public void exportTrainedModel​(java.nio.file.Path exportedFile)
                                throws java.io.IOException
        Exports the trained model binary file with model score threshold.
        Parameters:
        exportedFile - - destination binary model file.
        Throws:
        java.nio.file.FileAlreadyExistsException - if the given file already exists (optional).
        java.io.IOException - for various file copy errors
        java.lang.IllegalStateException - if trained model has not been saved for the active model.
        See Also:
        exportTrainedModel(Path, String)
      • exportModelAndQuery

        public void exportModelAndQuery​(int modelIndex,
                                        java.nio.file.Path exportedModelFile,
                                        java.lang.String modelVersion,
                                        java.nio.file.Path queryOutputDir,
                                        java.lang.String javaClassName,
                                        java.lang.String methodName)
                                 throws javax.xml.bind.JAXBException,
                                        java.io.IOException
        Exports all files needed to use the trained model on the Patterns server: the trained model binary file, Java source file that builds the Learn query, and any thesaurus files used by the query. Exports the files for any given saved model. Any previously obtained references to the active ModelSettings and related objects remain valid.
        Parameters:
        modelIndex - - the index of the model in the list of models (in ProjectSettings).
        exportedModelFile - - destination binary model file.
        modelVersion - - new version value for model file. If null, version is not assigned.
        queryOutputDir - - the directory where Java source file and thesaurus files are saved. Not null.
        javaClassName - - name of the Java class in the exported source. May include package name. If package is not included, the default package is used. Not null.
        methodName - - the name of the method that returns NetricsQuery in the exported Java source. Not null.
        Throws:
        javax.xml.bind.JAXBException - when XML validation fails, or if an XML file was created with newer Learn API. Previous references to the active ModelSettings are not guaranteed to be valid in this case.
        java.nio.file.FileAlreadyExistsException - if the given file already exists (optional).
        java.io.FileNotFoundException - if the model XML file for modelIndex is not found (optional).
        java.io.IOException - if I/O error occurs.
        java.lang.IllegalStateException - if trained model has not been saved for the active model.
        java.lang.IllegalArgumentException - if javaClassName is invalid.