Abstract
|
Sometimes, for example if the current index is corrupted and no backups are available, we have to rebuild the index. If the current index is still running and in use, i.e., it still works in some level, we cannot re-index it. For these situations, we have to make a parallel index that is indexed and after indexing taken into use.
Make sure that everybody talks about the same thing. This document is about number 1. Parallel, different indexes. |
Steps to create a parallel IDOL index in M-Files 20.5 and newer
1. Open admin tool and go to the corresponding vault > Configurations > Advanced Vault Settings > Configuration > Search > Indexes.
2. Click Add Index.
3. Open the new index node by clicking a small arrow left to the Index text.
4. Give the index Name and select Service Type: IDOL.
5. Open IDOL Options node.
6. Select IDOL Server Version (usually V12).
7. Set IDOL Server Host and ports:
-
IDOL Server ACI Port: ACI port of Daily engine or in single engine installation the one you are using. By default, it is 9000 (usually ending 0). You can check the right port from IDOL installation folder, usually in path like: C:\IDOL12\bin\IDOL_dirs-content-9300\content\content.cfg.
-
IDOL Server Indexing Port: Indexing port of Daily engine or in single engine installation the one you are using. By default, it is 9001 (usually ending 1).
-
Distributed ACI Port For DAH: In multi-engine installation ("distributed", more usual than single installation) this is ACI port for Distributed Action Handler component orchestrating searches. By default, 19000, but you can check it from a path like: C:\IDOL12\bin\IDOL_dirs-dahdih-19000\dah\dah.cfg.
-
Distributed ACI Port For DIH: This is ACI port for Distributed Index Handler component orchestrating indexing to the Main Index (vs. Daily). By default, 19003 (ending 3 since 0 is already reserved for DAH).
-
Distributed Indexing Port: This is indexing port DIH. By default, 19001 (ending 1).
8. Optional: Since the index is built from scratch, you might want to get it done faster. For that, open Additional Options node. There is a setting Number of Parallel Indexers. By default, we obviously need at least one index to index anything, but for times like this we can increase the number temporarily. Note, that each indexer uses some 12% of CPU.
Further note, in M-Files 21.x and beyond the combined/shared indexing mode combines the vault and the external repositories into one index, but for each external repository there will be individual indexer process. That said, if you have external repositories, the final number of indexers is (the number of repositories + 1 (the vault)) * the number of parallel indexers. For example, if you have the vault and two external repositories, all in combined mode and you add a new index with 2 parallel indexers, you will have 6 indexers overall. 9. Click Save.
Image 1. Basic configuration of IDOL cluster.
If everything was configured as it should, server status light on the bottom should go green or yellow and indexing starts automatically. You can see that from the Task Manager's Details tab. If you acc a column Command Line, you will see which MFIndexer.exe process belongs to which index.
When is the initial indexing round finished?
1. In the admin tool, open the Indexing Status section.
2. Select the corresponding Vault node > the first Partition node > New or Modified node. 3. Check Indexing Queue Size in Total. That will give you an estimate about how many objects are in the indexing queue. When the indexing is ready, that should be 0.
Image 3: Initial indexing is finished, when Indexing Queue Size in Total is 0.
Starting to use the new index for searches
When the initial round of indexing is ready, you can change M-Files to use the new index for the searches by changing the value of "ActiveCombinedIndex" into the name of the new index. Note that you can use only one index at a time for the searches.
Image 4. Changing M-Files to use the new index in the searches.
Steps to create a parallel IDOL index in M-Files 20.4 and older
1. Stop indexing manager (MFIndexingManager service) in M-Files server.
2. Copy and rename the IDOL index folder (usually named "Combined") in path C:\Program Files\M-Files\Server Vaults\Indexes. Rename it to the same as the new index registry name.
3. Remove the contents of the folder including subfolders you just created in 2.
4. Copy the IDOL index settings in from M-Files server's registry. The easiest way to do that is to Export the index key branch (in my case C_B91D75EF-A924-4583-A7BA-2A8F3C5614C3).
5. Modify the exported .reg file. Modify index registry name into something new (e.g. into C_B91D75EF-A924-4583-A7BA-2A8F3C5614C3_new). Then, import (double click) the file to the registry.
6. Go back to the registry (e.g. C_B91D75EF-A924-4583-A7BA-2A8F3C5614C3_new) and add RegSz, named as "Path". The value is the path to the corresponding index folder e.g. C:\Program Files\M-Files\Server Vaults\Indexes\C_B91D75EF-A924-4583-A7BA-2A8F3C5614C3_new
7. Start indexing manager > M-Files starts to index the new index.
8. After the indexing is finished, there will be two-time stamps in the file IndexRebuild.txt located in M-Files folder that was emptied in the number 6. The latter timestamp is the time indexing was finalized.
Alternatively, you can check the M-Files indexing queue in the following way. Note that the instructions below are for IDOL 10.9. IDOL 12.x differs in a way it only includes one file IndexCLog.log.
a. MetaData: Open file IndexMLog.log inside a folder "M-Files" in M-Files server's IDOL index folder (e.g. C:\Program Files\M-Files\Server Vaults\Indexes\combined\M-Files) and get the number from values parameter.
b. Run following query in SQL Server Management Studio against M-Files database:
SELECT count(*)
FROM [Sample Vault].[dbo].[OBJECTTYPEITEM]
where [VERSIONFORMDI] > number you got from the IndexMLog.log
The result tells how many objects are in the queue to index MetaData.
c. FileData: Open IndexFLog.log from the same location and get the number.
d. Run following query in M-Files database:
SELECT count(*)
FROM [Sample Vault].[dbo].[DOCUMENTFILE]
where [VERSIONFORFDI] > number you got from the IndexFLog.log
The result tells how many objects are in the queue to index FileData. Note, that this might show 0 if the FileData indexing is not started yet. With configuration in IDOL 10.9 the MetaData is indexed first and then the FileData. In IDOL 12.x MetaData and FileData are being indexed simultaneously, so there is only one file to check: IndecCLog.log.
9. When you want to start to use the new index, change "ActiveCombinedIndex" to the new (e.g. ActiveCombinedIndex (reg_sz) = C_B91D75EF-A924-4583-A7BA-2A8F3C5614C3_new) and take the vault offline and back to online (located in HKEY_LOCAL_MACHINE\SOFTWARE\Motive\M-Files\[version]\Server\MFServer\VaultOptions\[vault GUID]).
