org.forgerock / org.forgerock.opendj

06 Dec, 2013

2 commits

05 Dec, 2013

4 commits

02 Dec, 2013

3 commits

  • 186d2ac4   Renamed ds-cfg-compute-changenumber to ds-cfg-compute-change-number. ... Browse Dir » 
    
git-svn-id: https://svn.forgerock.org/opendj/trunk@9882 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • d91db94c   OPENDJ-1174 (CR-2631) Transfer responsibility for populating the ChangeNumberIndexDB to ChangelogDB ... Browse Dir » 
    
Hooked the ChangeNumberIndexer thread into our code base.
Managed thread lifecycle.
Added configuration to turn it on / off.
Removed code that builds the ChangeNumberIndexDB from ECLServerHandler.


ChangeNumberIndexer.java:
Added clear() + doClear field for unit tests
Used thread safe versions of MultiDomainServerState.
In run(), merged all the try blocks to factorize the catch blocks + supported doClear + fixed a bug with the mediumConsistencyRUV.
In removeCursor(), closed the cursors.
Added removeAllCursors(), getPrecedingCSN().
Renamed crossDomainDBCursor to nextChangeForInsertDBCursor and newCompositeDBCursor() to resetNextChangeForInsertDBCursor().
Changed newCursors from ConcurrentMap<CSN, DN> to ConcurrentMap<Pair<DN, Integer>, CSN> to ensure minimum memory consumption.

JEChangelogDB.java:
Removed dbDirectoryName field.
Added config, cnIndexer fields.
Changed ctor to accept ReplicationServerCfg.
In initializeDB(), shutdownDB(), clearDB(), publishUpdateMsg() took appropriate actions with the cnIndexer thread.
Added setComputeChangeNumber().
In getCursorFrom(), accept null parameter.
In setPurgeDelay(), also call it on the cnIndexDB.


MultiDomainServerState.java:
Made implementation to be fully thread safe.
Changed list from Map to ConcurrentMap + removed synchrnoized blocks.
Renamed update(DN, ServerState) to replace(DN, ServerState).
Added new version of update(DN, ServerState).

ECLServerHandler.java:
Removed assignNewChangeNumberAndStore() that builds the ChangeNumberIndexDB + adapted the code that was using this method.
Used MultiDomainServerState.replace().


ReplicationServerConfiguration.xml, ReplicationServerCfgDefn.properties:
Added ds-cfg-compute-changenumber.

ExternalChangelogDomainConfiguration.xml, ExternalChangelogDomainCfgDefn.properties:
Updated description

ReplicationServer.java:
In applyConfigurationChange(), handled computeChangenumber config.
Used MultiDomainServerState.replace().

ChangelogDB.java:
Added setComputeChangeNumber().

ReplServerFakeConfiguration.java:
Added isComputeChangenumber() and setComputeChangenumber().


ExternalChangeLogTest.java:
Adapted the code to the use of the ChangeNumberIndexer thread.

JEChangeNumberIndexDB.java:
Extracted runPurge() from run().


ReplicationDbEnv.java
Code cleanup.


git-svn-id: https://svn.forgerock.org/opendj/trunk@9881 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 6dc55f73   Fixed unit test. ... Browse Dir » 
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9877 41b1ffd8-f28e-4786-ab96-9950f0a78031
    violette
     

28 Nov, 2013

2 commits

  • ac1c864e   Code cleanup. ... Browse Dir » 
    ExternalChangeLogTest.java:
In searchOnChangelog(), added the parameters expectedNbEntries and testName + added a loop that tries during 1s to get the expected number of entries + moved assert on nb of returned entries here + moved calls to debugAndWriteEntries() here.
Removed now unnecessary calls to Thread.sleep().
Added assertSuccessful(LDAPMessage).
Reduced scope for variables of type SearchResultDoneProtocolOp or SearchResultEntryProtocolOp + removed now useless statements making use of them.
In ECLCompatWriteReadAllOps(), removed useless calls to toLowercase().
Used Assertions.assertThat().

git-svn-id: https://svn.forgerock.org/opendj/trunk@9857 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 625b9927   Code cleanup ... Browse Dir » 
    Added toString() for debuggging.
Added or completed javadocs.
Fixed a little bug in test code.

git-svn-id: https://svn.forgerock.org/opendj/trunk@9852 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

23 Nov, 2013

3 commits

22 Nov, 2013

3 commits

  • 7626a24a   Code cleanup. ... Browse Dir » 
    ExternalChangeLogTest.java:
In getAttributeValue(), added null check.
Removed getAttributeValueOrNull() replaced by getAttributeValue().
Removed checkPossibleValues(), replaced by checkValue().
In readLastCookie(), called getAttributeValue() and removed useless catch.
Extracted method checkAttributeValue().Used Assertions.as() - thanks Nicolas for the trick.


git-svn-id: https://svn.forgerock.org/opendj/trunk@9831 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 7acdc477   Checkpoint commit for OPENDJ-1174 Transfer responsibility for populating the Cha… ... Browse Dir » 
    …ngeNumberIndexDB to ChangelogDB


Properly implemented the medium consistency point algorithm with heartbeats and replicas going offline.


ChangeNumberIndexer.java:
Added replicasOffline instance field.
Added methods replicaOffline() and removeCursor().
In moveForwardMediumConsistencyPoint(), updated the code to cater for offline replica.

ChangeNumberIndexerTest.java:
Added new tests for heartbeat and replica going offline.


MultiDomainServerState.java, ServerState.java:
Added removeCSN().

MultiDomainServerStateTest.java, ServerStateTest.java:
Added more tests.

git-svn-id: https://svn.forgerock.org/opendj/trunk@9828 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • bfed8cb9   Checkpoint commit for OPENDJ-1174 Transfer responsibility for populating the Cha… ... Browse Dir » 
    …ngeNumberIndexDB to ChangelogDB


Properly implemented the medium consistency point algorithm.


ChangeNumberIndexer.java:
Renamed instance field previousCookie to mediumConsistencyRUV and mediumConsistencyPoint to lastSeenUpdates.
Added mediumConsistencyCSN instance field.
Changed newCursors from ConcurrentMap<Integer, DN> to ConcurrentMap<CSN, DN>.
In tryNotify(), removed CSN parameter.
Added methods canMoveForwardMediumConsistencyPoint() and moveForwardMediumConsistencyPoint().


ChangeNumberIndexerTest.java:
Updated to reflect how the medium consistency point algorithm should work.

MultiDomainServerState.java:
Renamed get() to getCSN().

ServerState.java
In update(), updated javadocs + renamed local variables.

git-svn-id: https://svn.forgerock.org/opendj/trunk@9827 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

21 Nov, 2013

3 commits

  • 284fcab7   If a server fails to stop, kill it to avoid impact on other testcases ... Browse Dir » 
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9824 41b1ffd8-f28e-4786-ab96-9950f0a78031
    csovant
     
  • b956d9ba   Checkpoint commit for OPENDJ-1174 Transfer responsibility for populating the Cha… ... Browse Dir » 
    …ngeNumberIndexDB to ChangelogDB 

Fixed imports for Pair class which has been moved recently.

git-svn-id: https://svn.forgerock.org/opendj/trunk@9821 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • ac2279e0   Checkpoint commit for OPENDJ-1174 Transfer responsibility for populating the Cha… ... Browse Dir » 
    …ngeNumberIndexDB to ChangelogDB 



ChangeNumberIndexer.java, ChangeNumberIndexerTest.java: ADDED

CompositeDBCursor.java:
Converted this class from holding a Collection of DBCursor to holding a Map of DBCursor with their associated data + generecized it.
Added getData().

CompositeDBCursorTest.java:
Consequence of the change to CompositeDBCursor.
Moved MyDBCursor inner class to its own file.

SequentialDBCursor.java: ADDED
Extracted from CompositeDBCursorTest.MyDBCursor.

ReplicationDomainDB.java:
Pulled up getCursorFrom(DN, int, CSN) in order to be able to mock it.

MultiDomainServerState.java:
In update(DN, CSN), simplified the code.
In update(DN, serverState), made the code better.
Added update(MultiDomainServerState), get(DN) and cover(DN, CSN).

AttributeTypeConstants.java:
Added support for 'o', 'ou' and 'dc'.

JEReplicaDBCursor.java:
In toString(), fixed display.

git-svn-id: https://svn.forgerock.org/opendj/trunk@9820 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

19 Nov, 2013

1 commit

  • 204e0dc1   ServerStateTest.java: ... Browse Dir » 
    Added @Test annotation to testCover() - follow up of r9800

ServerState.java:
Little code cleanup.

git-svn-id: https://svn.forgerock.org/opendj/trunk@9811 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

15 Nov, 2013

1 commit

14 Nov, 2013

4 commits

13 Nov, 2013

2 commits

  • 6c76e6d7   Complement of r9789 ... Browse Dir » 
    ExternalChangeLogTest.java:
Put the actual expected message.

git-svn-id: https://svn.forgerock.org/opendj/trunk@9793 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • de389c70   replication*.properties: ... Browse Dir » 
    Added more info for ERR_INVALID_COOKIE_SYNTAX.

MultiDomainServerState.java:
Consequence of the change to ERR_INVALID_COOKIE_SYNTAX.
In splitGenStateToServerStates(), code clean up.

ExternalChangeLogTest.java:
Consequence of the change to ERR_INVALID_COOKIE_SYNTAX.
Used Assertions.

git-svn-id: https://svn.forgerock.org/opendj/trunk@9789 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

12 Nov, 2013

2 commits

  • eea0b339   Staf refactor ... Browse Dir » 
    TODO: Added
Starting a virtuous cycle where Christophe can put tasks for cleaning up the staf-tests by adding a line in this file.
This file already contains instructions on how to start doing some cleanup tasks to reduce the size of STAF test cases.

git-svn-id: https://svn.forgerock.org/opendj/trunk@9787 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 12bc1e79   CR-2563 JEChangelogDB removing a synchronized block in a heavily hit path ... Browse Dir » 
    JEChangelogDB.java:
Used ConcurrentMap for domainToReplicaDBs.
Added shutdown field to better protect various code paths against shutdown.
In getOrCreateReplicaDB(), removed global synchronization block with fined grained synchronization code that is only used on JEReplicaDB creation.
Added getExistingOrNewDomainMap(), getExistingOrNewReplicaDB().
Extracted innerShutdownDomain() from shutdownDomain().
In shutdownDB(), ensured shutdown of all the replicaDBs.
In shutdownDB(), shutdownDomain() and getOrCreateReplicaDB() check whether the DB is shutting down.
In removeDomain() and innerShutdownDomain() ensured the domainMap is removed before shutting down the replicaDBs.

replication.properties:
Added an error message to be used when the DB is shutting down.
Extracted and separated method init() from JEReplicaDB ctor.

JEReplicaDB.java:
In ctor, renamed id parameter to serverId + removed queueSize parameter.

JEReplicaDBTest.java:
Consequence of the change to JEReplicaDB.

git-svn-id: https://svn.forgerock.org/opendj/trunk@9786 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

06 Nov, 2013

1 commit

  • 49010f5e   OPENDJ-1053 Improve logging of replication load balancing and fail-over in order… ... Browse Dir » 
    … to diagnose the causes of these events



Functional tests found a ConcurrentModificationException with r9753 (see at the end of the commit message):
The code was removing elements from a Collection while it was iterating on the exact same Collection.



ReplicationBroker.java:
In inner class LocalEvaluation:
- Renamed "filteredRSs" to "accepted".
- Added getAcceptedRSInfos() to allow iterating on them without triggering ConcurrentModificationException.
In rejectAll*(), called getAcceptedRSInfos().
In getCSN(), renamed parameters.

Fixed ConcurrentModificationException.

ComputeBestServerTest.java:
Added a test2ServersUpToDateAnd1EvenMoreUpToDate() to exhibit the ConcurrentModificationException.
Extracted method newServerState() and used it in all tests.
In all test methods but one, do not update the ServerState with unused CSNs because they make the tests harder to read. Did not change test3Servers() because I believe it is good to keep one test testing the unneded CSNs do not impact the test results.




java.util.ConcurrentModificationException
	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
	at java.util.HashMap$ValueIterator.next(HashMap.java:822)
	at org.opends.server.replication.service.ReplicationBroker.rejectAllWithRSOnDifferentVMThanDS(ReplicationBroker.java:1908)
	at org.opends.server.replication.service.ReplicationBroker.filterServersOnSameHost(ReplicationBroker.java:1878)
	at org.opends.server.replication.service.ReplicationBroker.computeBestReplicationServer(ReplicationBroker.java:1600)
	at org.opends.server.replication.service.ReplicationBroker.connectAsDataServer(ReplicationBroker.java:820)
	at org.opends.server.replication.service.ReplicationBroker.connect(ReplicationBroker.java:691)
	at org.opends.server.replication.service.ReplicationBroker.start(ReplicationBroker.java:221)
	at org.opends.server.replication.service.ReplicationDomain.enableService(ReplicationDomain.java:3024)
	at org.opends.server.replication.service.ReplicationDomain.changeConfig(ReplicationDomain.java:3054)
	at org.opends.server.replication.plugin.LDAPReplicationDomain.applyConfigurationChange(LDAPReplicationDomain.java:4089)
	at org.opends.server.replication.plugin.LDAPReplicationDomain.applyConfigurationChange(LDAPReplicationDomain.java:96)
	at org.opends.server.admin.server.ServerManagedObjectChangeListenerAdaptor.applyConfigurationChange(ServerManagedObjectChangeListenerAdaptor.java:74)
	at org.opends.server.admin.server.ConfigChangeListenerAdaptor.applyConfigurationChange(ConfigChangeListenerAdaptor.java:342)
	at org.opends.server.extensions.ConfigFileHandler.replaceEntry(ConfigFileHandler.java:1610)
	at org.opends.server.workflowelement.localbackend.LocalBackendModifyOperation.processModify(LocalBackendModifyOperation.java:599)
	at org.opends.server.workflowelement.localbackend.LocalBackendModifyOperation.processLocalModify(LocalBackendModifyOperation.java:299)
	at org.opends.server.workflowelement.localbackend.LocalBackendWorkflowElement.execute(LocalBackendWorkflowElement.java:690)
	at org.opends.server.core.WorkflowImpl.execute(WorkflowImpl.java:197)
	at org.opends.server.core.WorkflowTopologyNode.execute(WorkflowTopologyNode.java:100)
	at org.opends.server.core.ModifyOperationBasis.run(ModifyOperationBasis.java:451)
	at org.opends.server.core.SynchronousStrategy.enqueueRequest(SynchronousStrategy.java:49)
	at org.opends.server.protocols.ldap.LDAPClientConnection.addOperationInProgress(LDAPClientConnection.java:1276)
	at org.opends.server.protocols.ldap.LDAPClientConnection.processModifyRequest(LDAPClientConnection.java:2268)
	at org.opends.server.protocols.ldap.LDAPClientConnection.processLDAPMessage(LDAPClientConnection.java:1745)
	at org.opends.server.protocols.ldap.LDAPRequestHandler.run(LDAPRequestHandler.java:194)

git-svn-id: https://svn.forgerock.org/opendj/trunk@9784 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

04 Nov, 2013

2 commits

  • aed0a3a7   OPENDJ-1190 (CR-2523) Under rare circumstances the DS replication recovery threa… ... Browse Dir » 
    …d (RSUpdater) can spin


This change is linked to the misbehaving RSUpdater thread which (wrongly):
- could be started multiple times
- would not shutdown willingly when the server is shutting down
- would try to look for replay operations in the future



LDAPReplicationDomain.java:
In buildAndPublishMissingChanges(), only exit the loop when the currentStartCSN is newer than now (instead of the incorrect currentStartCSN + 10s).

HistoricalCsnOrderingTest.java:
Code cleanup.

git-svn-id: https://svn.forgerock.org/opendj/trunk@9767 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 9796b35c   OPENDJ-1190 (CR-2523) Under rare circumstances the DS replication recovery threa… ... Browse Dir » 
    …d (RSUpdater) can spin


This change is linked to the misbehaving RSUpdater thread which (wrongly):
- could be started multiple times
- would not shutdown willingly when the server is shutting down
- would try to look for replay operations in the future



LDAPReplicationDomain.java:
Added AtomicReference for RSUpdater thread.
Used DirectoryThread.isShutdownInitiated() for ServerStateFlush and RSUpdater threads.
In RSUpdater, added shutdown field to pass it down to buildAndPublishMissingChanges() + overrode initiateShutdown() to set the shutdown field to true.
In shutdown(), called initiateShutdown() for RSUpdater and RSUpdater threads.
In sessionInitiated(), only start the RSUpdaterThread if it does not already exist.
In buildAndPublishMissingChanges(), added early exits in case of shutdown.
Added now() method.

HistoricalCsnOrderingTest.java:
Consequence of the change to LDAPReplicationDomain.

git-svn-id: https://svn.forgerock.org/opendj/trunk@9764 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

29 Oct, 2013

2 commits

  • 79d7cb2c   Fixing failing tests in continuous integration. ... Browse Dir » 
    
AssuredReplicationServerTest.java
Fixed lost configuration lost in r9746.

FakeReplicationDomain.java, ReplicationDomainTest.java:
Added back configuration accidentally removed in r9746.

git-svn-id: https://svn.forgerock.org/opendj/trunk@9759 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 2c86336c   StateMachineTest.java: ... Browse Dir » 
    In createReplicationBroker(), added back configuration accidentally removed in r9746

git-svn-id: https://svn.forgerock.org/opendj/trunk@9758 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

28 Oct, 2013

2 commits

  • dcc0d28e   AssuredReplicationServerTest.java: ... Browse Dir » 
    Used Assertions to have more descriptive assertion failures.

git-svn-id: https://svn.forgerock.org/opendj/trunk@9754 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • ec480c97   OPENDJ-1053 (CR-2520) Improve logging of replication load balancing and fail-ove… ... Browse Dir » 
    …r in order to diagnose the causes of these events


This change records how each replication server is evaluated by computeBestReplicationServer().
It also logs why a DS switches from one RS to another RS.


ReplicationBroker.java:
Added static classes RSEvaluations and LocalEvaluation.
Moved keepBest() method to RSEvaluations.
In filterServersWithAllLocalDSChanges(), reworked the code to be more understandable + Added method getCSN() and rejectAllWithRSIsLaterThanBestRS(), rejectAllWithRSOnDifferentVMThanDS().
In computeBestServerWhenConnected(), extracted method isServerOverloadingRS().

replication.properties:
Added tons of messages.

ComputeBestServerTest.java:
Moved this class to package org.opends.server.replication.service to match the package of the tested code.
Consequence of the changes to ReplicationBroker.
Added methods containsOnly(), getDescription(), getEval1() and getEval2().
Reformatted the code a bit.
Removed @throws from javadocs.

git-svn-id: https://svn.forgerock.org/opendj/trunk@9753 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

23 Oct, 2013

1 commit

  • ac9a8c6f   LDAPReplicationDomain.java: ... Browse Dir » 
    Replaced instance fields with directly storing and using the ReplicationDomainCfg object.

ReplicationBroker.java:
Replaced instance fields with directly storing and using the ReplicationDomainCfg object.
Removed start(Set<String>), setChangeTimeHeartbeatInterval() (now useless), and setGroupId() (never used).
Added getBaseDN(), getReplicationServerUrls(), getGroupId().
In several methods, used Map.entrySet() instead of Map.keySet() with Map.get(Object)  + renamed replicationServerInfo to rsInfo.

replication.properties:
Removed now unused message.


ReplicationDomain.java:
Replaced instance fields with directly storing and using the ReplicationDomainCfg object.
Code cleanups:
- Removed useless use of "this." for method calls.
- In processErrorMsg() and processUpdateDone(), collapsed if statements
- Increased vertical density
- Extracted method getGenId()
- In waitForAckIfAssuredEnabled(), used early exit.


DomainFakeCfg.java:
Comments => javadocs
baseDn => baseDN
Removed several unused methods.
Added setWindowSize() + instance member.

*Test*.java
Used the DomainFakeCfg everywhere + SortedSet instead of Set.

git-svn-id: https://svn.forgerock.org/opendj/trunk@9746 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

17 Oct, 2013

2 commits

  • 3c108d77   Fixed GenerationIdTest.generationIdTest(): test was failing precisely because th… ... Browse Dir » 
    …e directory server's broker was not connected to a replication domain, hence it was impossible to add the changes to the changelogDB and then find them when querying the changelogDB.


GenerationIdTest.java:
Renamed assertConnectedToReplicationDomain() to waitConnectionToReplicationDomain().

LDAPReplicationDomain.java:
Removed uses of StringBuilder, relied on *Operation.toString() to do it for us.

ReplicationBroker.java:
Improved toString().

ReplicationTestCase.java:
Removed a wrong comment.
Used Assertions.assertThat().

ReplicationServerTest.java:
Removed a useless call to DirectoryServer.getSynchronizationProviders().

git-svn-id: https://svn.forgerock.org/opendj/trunk@9736 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 17bdce54   GenerationIdTest.generationIdTest() test is not fixed, but I have now determined… ... Browse Dir » 
    … that it is failing because the server is not connected to a replication domain, hence it is impossible to add the changes to the changelog.

GenerationIdTest.java:
Added assertConnectedToReplicationDomain() + called it.
Fixed a few logged strings.

git-svn-id: https://svn.forgerock.org/opendj/trunk@9735 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac