06 Nov, 2013

1 commit

  • … to diagnose the causes of these events
    
    
    
    Functional tests found a ConcurrentModificationException with r9753 (see at the end of the commit message):
    The code was removing elements from a Collection while it was iterating on the exact same Collection.
    
    
    
    ReplicationBroker.java:
    In inner class LocalEvaluation:
    - Renamed "filteredRSs" to "accepted".
    - Added getAcceptedRSInfos() to allow iterating on them without triggering ConcurrentModificationException.
    In rejectAll*(), called getAcceptedRSInfos().
    In getCSN(), renamed parameters.
    
    Fixed ConcurrentModificationException.
    
    ComputeBestServerTest.java:
    Added a test2ServersUpToDateAnd1EvenMoreUpToDate() to exhibit the ConcurrentModificationException.
    Extracted method newServerState() and used it in all tests.
    In all test methods but one, do not update the ServerState with unused CSNs because they make the tests harder to read. Did not change test3Servers() because I believe it is good to keep one test testing the unneded CSNs do not impact the test results.
    
    
    
    
    java.util.ConcurrentModificationException
    	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
    	at java.util.HashMap$ValueIterator.next(HashMap.java:822)
    	at org.opends.server.replication.service.ReplicationBroker.rejectAllWithRSOnDifferentVMThanDS(ReplicationBroker.java:1908)
    	at org.opends.server.replication.service.ReplicationBroker.filterServersOnSameHost(ReplicationBroker.java:1878)
    	at org.opends.server.replication.service.ReplicationBroker.computeBestReplicationServer(ReplicationBroker.java:1600)
    	at org.opends.server.replication.service.ReplicationBroker.connectAsDataServer(ReplicationBroker.java:820)
    	at org.opends.server.replication.service.ReplicationBroker.connect(ReplicationBroker.java:691)
    	at org.opends.server.replication.service.ReplicationBroker.start(ReplicationBroker.java:221)
    	at org.opends.server.replication.service.ReplicationDomain.enableService(ReplicationDomain.java:3024)
    	at org.opends.server.replication.service.ReplicationDomain.changeConfig(ReplicationDomain.java:3054)
    	at org.opends.server.replication.plugin.LDAPReplicationDomain.applyConfigurationChange(LDAPReplicationDomain.java:4089)
    	at org.opends.server.replication.plugin.LDAPReplicationDomain.applyConfigurationChange(LDAPReplicationDomain.java:96)
    	at org.opends.server.admin.server.ServerManagedObjectChangeListenerAdaptor.applyConfigurationChange(ServerManagedObjectChangeListenerAdaptor.java:74)
    	at org.opends.server.admin.server.ConfigChangeListenerAdaptor.applyConfigurationChange(ConfigChangeListenerAdaptor.java:342)
    	at org.opends.server.extensions.ConfigFileHandler.replaceEntry(ConfigFileHandler.java:1610)
    	at org.opends.server.workflowelement.localbackend.LocalBackendModifyOperation.processModify(LocalBackendModifyOperation.java:599)
    	at org.opends.server.workflowelement.localbackend.LocalBackendModifyOperation.processLocalModify(LocalBackendModifyOperation.java:299)
    	at org.opends.server.workflowelement.localbackend.LocalBackendWorkflowElement.execute(LocalBackendWorkflowElement.java:690)
    	at org.opends.server.core.WorkflowImpl.execute(WorkflowImpl.java:197)
    	at org.opends.server.core.WorkflowTopologyNode.execute(WorkflowTopologyNode.java:100)
    	at org.opends.server.core.ModifyOperationBasis.run(ModifyOperationBasis.java:451)
    	at org.opends.server.core.SynchronousStrategy.enqueueRequest(SynchronousStrategy.java:49)
    	at org.opends.server.protocols.ldap.LDAPClientConnection.addOperationInProgress(LDAPClientConnection.java:1276)
    	at org.opends.server.protocols.ldap.LDAPClientConnection.processModifyRequest(LDAPClientConnection.java:2268)
    	at org.opends.server.protocols.ldap.LDAPClientConnection.processLDAPMessage(LDAPClientConnection.java:1745)
    	at org.opends.server.protocols.ldap.LDAPRequestHandler.run(LDAPRequestHandler.java:194)
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9784 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

05 Nov, 2013

3 commits

  • In createChangelogEntry(), removed duplicate parameters.
    In createEntryFromMsg(), extracted method addAttribute().
    Changed returnECLControl field to boolean (incorrectly was Boolean)
    Improved formatting.
    Removed comments paraphrasing the code.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9777 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Added complimentary javadocs to explain what these constants mean.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9776 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Renamed variables and changed comments to match the new terminology.
    Added more comments to explain what the code is doing.
    
    ReplicationBroker.java
    Renamed replicationServerInfo => rsInfo.
    
    replication.properties
    Removed space characters at the end of each lines.
    
    RSInfo.java:
    In toString(), removed newline + improved formatting.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9775 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

04 Nov, 2013

5 commits

  • …d (RSUpdater) can spin
    
    
    This change is linked to the misbehaving RSUpdater thread which (wrongly):
    - could be started multiple times
    - would not shutdown willingly when the server is shutting down
    - would try to look for replay operations in the future
    
    
    
    LDAPReplicationDomain.java:
    In now(), ensured now() will always come last with isNewerThan() test.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9769 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • …d (RSUpdater) can spin
    
    
    This change is linked to the misbehaving RSUpdater thread which (wrongly):
    - could be started multiple times
    - would not shutdown willingly when the server is shutting down
    - would try to look for replay operations in the future
    
    
    
    LDAPReplicationDomain.java:
    In buildAndPublishMissingChanges(), only exit the loop when the currentStartCSN is newer than now (instead of the incorrect currentStartCSN + 10s).
    
    HistoricalCsnOrderingTest.java:
    Code cleanup.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9767 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Updated retrieving the source jars.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9766 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • git-svn-id: https://svn.forgerock.org/opendj/trunk@9765 41b1ffd8-f28e-4786-ab96-9950f0a78031
    cjr
     
  • …d (RSUpdater) can spin
    
    
    This change is linked to the misbehaving RSUpdater thread which (wrongly):
    - could be started multiple times
    - would not shutdown willingly when the server is shutting down
    - would try to look for replay operations in the future
    
    
    
    LDAPReplicationDomain.java:
    Added AtomicReference for RSUpdater thread.
    Used DirectoryThread.isShutdownInitiated() for ServerStateFlush and RSUpdater threads.
    In RSUpdater, added shutdown field to pass it down to buildAndPublishMissingChanges() + overrode initiateShutdown() to set the shutdown field to true.
    In shutdown(), called initiateShutdown() for RSUpdater and RSUpdater threads.
    In sessionInitiated(), only start the RSUpdaterThread if it does not already exist.
    In buildAndPublishMissingChanges(), added early exits in case of shutdown.
    Added now() method.
    
    HistoricalCsnOrderingTest.java:
    Consequence of the change to LDAPReplicationDomain.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9764 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

30 Oct, 2013

2 commits

  • In runTask(), improved error logging in case of exceptions.
    Little code cleanup.
    
    TaskThread.java
    Logged the task state after the task finished.
    Comments => javadocs.
    
    backend*.properties:
    Updated NOTICE_TASK_FINISHED_414 message. Updated the translations where I could, otherwise removed them (alas).
    
    replication.properties, LDAPReplicationDomain.java:
    Removed useless spaces in error logging
    
    DSInfo.java:
    In toString(), removed newline + improved formatting.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9762 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Added cloneWithReplicationServerId().
    
    LightweightServerHandler.java:
    Simplified ctor by just accepting a DSInfo object.
    Extracted method debugInfo()
    Removed redundant fields.
    
    ReplicationServerHandler.java:
    In processTopoInfoFromRS(), called DSInfo.cloneWithReplicationServerId() + consequence of the change to LightweightServerHandler.
    
    ReplicationDomain.java:
    Small code cleanup.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9761 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

29 Oct, 2013

2 commits

  • 
    AssuredReplicationServerTest.java
    Fixed lost configuration lost in r9746.
    
    FakeReplicationDomain.java, ReplicationDomainTest.java:
    Added back configuration accidentally removed in r9746.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9759 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • In createReplicationBroker(), added back configuration accidentally removed in r9746
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9758 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

28 Oct, 2013

3 commits

  • replication.properties
    Removed the forbidden final dot from i18n message.
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9755 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Used Assertions to have more descriptive assertion failures.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9754 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • …r in order to diagnose the causes of these events
    
    
    This change records how each replication server is evaluated by computeBestReplicationServer().
    It also logs why a DS switches from one RS to another RS.
    
    
    ReplicationBroker.java:
    Added static classes RSEvaluations and LocalEvaluation.
    Moved keepBest() method to RSEvaluations.
    In filterServersWithAllLocalDSChanges(), reworked the code to be more understandable + Added method getCSN() and rejectAllWithRSIsLaterThanBestRS(), rejectAllWithRSOnDifferentVMThanDS().
    In computeBestServerWhenConnected(), extracted method isServerOverloadingRS().
    
    replication.properties:
    Added tons of messages.
    
    ComputeBestServerTest.java:
    Moved this class to package org.opends.server.replication.service to match the package of the tested code.
    Consequence of the changes to ReplicationBroker.
    Added methods containsOnly(), getDescription(), getEval1() and getEval2().
    Reformatted the code a bit.
    Removed @throws from javadocs.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9753 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

25 Oct, 2013

1 commit


23 Oct, 2013

2 commits

  • Thanks to Chris Ridd for reviewing the patch I posted to the issue and letting me know by IM.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9747 41b1ffd8-f28e-4786-ab96-9950f0a78031
    mark
     
  • Replaced instance fields with directly storing and using the ReplicationDomainCfg object.
    
    ReplicationBroker.java:
    Replaced instance fields with directly storing and using the ReplicationDomainCfg object.
    Removed start(Set<String>), setChangeTimeHeartbeatInterval() (now useless), and setGroupId() (never used).
    Added getBaseDN(), getReplicationServerUrls(), getGroupId().
    In several methods, used Map.entrySet() instead of Map.keySet() with Map.get(Object)  + renamed replicationServerInfo to rsInfo.
    
    replication.properties:
    Removed now unused message.
    
    
    ReplicationDomain.java:
    Replaced instance fields with directly storing and using the ReplicationDomainCfg object.
    Code cleanups:
    - Removed useless use of "this." for method calls.
    - In processErrorMsg() and processUpdateDone(), collapsed if statements
    - Increased vertical density
    - Extracted method getGenId()
    - In waitForAckIfAssuredEnabled(), used early exit.
    
    
    DomainFakeCfg.java:
    Comments => javadocs
    baseDn => baseDN
    Removed several unused methods.
    Added setWindowSize() + instance member.
    
    *Test*.java
    Used the DomainFakeCfg everywhere + SortedSet instead of Set.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9746 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

21 Oct, 2013

1 commit


17 Oct, 2013

4 commits

  • 
    ReplicationDbEnv.java, replication.properties:
    In openDatabase(), fixed TODOs for i18n.
    In newErrorMessage(), improved error logging by including the DB name.
    
    JEChangelogDB.java:
    Simplified the code by calling domainToReplicaDBs.remove() from inside shutdownReplicaDBs().
    In shutdownReplicaDBs(), do not bother cleaning up the domainMap since it will be removed from domainToReplicaDBs.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9737 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • …e directory server's broker was not connected to a replication domain, hence it was impossible to add the changes to the changelogDB and then find them when querying the changelogDB.
    
    
    GenerationIdTest.java:
    Renamed assertConnectedToReplicationDomain() to waitConnectionToReplicationDomain().
    
    LDAPReplicationDomain.java:
    Removed uses of StringBuilder, relied on *Operation.toString() to do it for us.
    
    ReplicationBroker.java:
    Improved toString().
    
    ReplicationTestCase.java:
    Removed a wrong comment.
    Used Assertions.assertThat().
    
    ReplicationServerTest.java:
    Removed a useless call to DirectoryServer.getSynchronizationProviders().
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9736 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • … that it is failing because the server is not connected to a replication domain, hence it is impossible to add the changes to the changelog.
    
    GenerationIdTest.java:
    Added assertConnectedToReplicationDomain() + called it.
    Fixed a few logged strings.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9735 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • In toString(), improved the returned String.
    Changed rsServerId field from Integer to int.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9731 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

16 Oct, 2013

1 commit

  • Used StaticUtils.stackTraceToSingleLineString(e)) for logging on one line only.
    
    ReplicationBackend.java:
    Renamed all *containers to *domains.
    Reworked findExportContainers() and findSearchContainers() into only one method: selectReplicationDomains().
    
    ListenerThread.java:
    Fixed formatting.
    
    ReplicationServerLoadBalancingTest.java:
    Code cleanup.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9729 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

15 Oct, 2013

3 commits

  • More code cleanup.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9728 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Code cleanup.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9727 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • UpdateOperationTest.java:
    Added consumeAllMessages(ReplicationBroker).
    In csnGeneratorAdjust(), called consumeAllMessages().
    Extracted local variables serverId.
    Fixed formatting.
    Increased vertical density.
    Removed a useless call to Thread.sleep().
    Removed useless comments.
    
    ReplicationBroker.java:
    Used local variables "localSession" to avoid concurrency issues when setSession(null) is called by another thread.
    
    *.java:
    Implemented toString().
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9725 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

14 Oct, 2013

1 commit

  • Tried to simplify the code + separated similar looking test data between the different tests to help debug the (not so) random test failure.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9724 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

12 Oct, 2013

1 commit


11 Oct, 2013

5 commits

  • 
    ReplicationTestCase.java:
    In executeTask() and waitTaskState() added maxWaitTimeInMillis parameter.
    
    HistoricalTest.java:
    In testRecurringPurgeIn1Run(), added a sleep before configuring conflicts purge delay + waited purge for a maximum of 120 seconds (was approximatively 20 seconds before): as long as task max duration.
    
    InitOnLineTest.java, ReplicationServerTest.java:
    Consequence of the change to ReplicationTestCase.
    
    GenerationIdTest.java:
    Consequence of the change to ReplicationTestCase.
    Extracted method createSetGenerationIdTask().
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9719 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • In computeInitialServerStatus(), used early exits.
    In computeBestServerForWeight(), extracted methods computeBestServerWhenNotConnected() and computeBestServerWhenConnected().
    Changed replicationServerUrls from Collection<String> to Set<String>.
    Removed useless field initialization to null.
    Renamed _publish() to publish() + reduced local variables scope.
    In receive(), renamed local variable replicationServerID to previousRsServerID and used this one more rather than the field.
    In changeConfig(), used Set.equals().
    Changed getReplicationMonitor() to getReplicationMonitorInstanceName().
    
    ReplicationDomain.java:
    Consequence of the change to ReplicationBroker.getReplicationMonitor(), inlined getReplicationMonitorInstanceName().
    Consequence of the change to ReplicationBroker.replicationServerUrls.
    
    TestCaseUtils.java:
    Moved newSet(), newSortedSet(), newList() here from test classes.
    
    *Test.java:
    Consequence of the changes to ReplicationBroker.replicationServerUrls.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9712 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • package org.opends.server.types requires javadoc even for private methods.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9711 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • RDN.java:
    In getDNValue(), changed parameter from String to ByteString.
    In compareTo(), reordered the if statements + extracted method compare().
    Comments => javadocs
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9710 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Fixed a regression introduced in r9605:  It was found by Christophe Sovant with the functional test suite.
    
    Partial revert of r9605:
    Using DN.toNormalizedString() should have been only done for replication metadata, not for user data because OpenDJ stores and returns data like provided by users.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9709 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

10 Oct, 2013

5 commits

  • Fixed a regression introduced in r9677. It was found by Christophe Sovant with the functional test suite:
    When throwing a DirectoryException with ERR_RESYNC_REQUIRED_UNKNOWN_DOMAIN_IN_PROVIDED_COOKIE, the domainCtxts is read before it is written to.
    This code should have been changed to iterate on the Set<DomainContext> rather than the domainCtxts field which has not been assigned at this point.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9707 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • git-svn-id: https://svn.forgerock.org/opendj/trunk@9706 41b1ffd8-f28e-4786-ab96-9950f0a78031
    csovant
     
  • DBCursor.java:
    Added usage example to the javadocs.
    
    DraftCNDB.java:
    Replaced catch (Exception) by catch (DatabaseException), or removed them altogether where unapplicable.
    In next(), first wiped the current record + reverted the if condition and clauses.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9704 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Changes after review from Matthew Swift.
    
    Renamed CNIndexRecord to ChangeNumberIndexRecord to be consistent with ChangeNumberIndexDB name.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9703 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 
    Removed ChangeNumberIndexDB.getRecord(long).
    
    
    ECLServerHandler.java:
    Replaced calls to ChangeNumberIndexDB.getRecord() by calls to ChangeNumberIndexDB.getCursorFrom().
    Inlined all the crossDomainStartState local variables.
    Added getCursorFrom() to preserve old behaviour from JEChangeNumberIndexDBCursor ctor.
    
    
    ChangeNumberIndexDB.java:
    Removed getRecord().
    
    JEChangeNumberIndexDB.java:
    Removed getReadCursor(), getRecord() + some surprising and never used code like lock field, hasLock(), lock() and release().
    
    JEChangeNumberIndexDBCursor.java:
    In ctor, do not throw ChangelogException if the changeNumber cannot be found.
    
    JEChangeNumberIndexDBTest.java:
    Used JEChangeNumberIndexDB.getCursorFrom() instead of JEChangeNumberIndexDB.getReadCursor().
    Renamed a few variables.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9702 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac