12 Nov, 2013

1 commit

  • JEChangelogDB.java:
    Used ConcurrentMap for domainToReplicaDBs.
    Added shutdown field to better protect various code paths against shutdown.
    In getOrCreateReplicaDB(), removed global synchronization block with fined grained synchronization code that is only used on JEReplicaDB creation.
    Added getExistingOrNewDomainMap(), getExistingOrNewReplicaDB().
    Extracted innerShutdownDomain() from shutdownDomain().
    In shutdownDB(), ensured shutdown of all the replicaDBs.
    In shutdownDB(), shutdownDomain() and getOrCreateReplicaDB() check whether the DB is shutting down.
    In removeDomain() and innerShutdownDomain() ensured the domainMap is removed before shutting down the replicaDBs.
    
    replication.properties:
    Added an error message to be used when the DB is shutting down.
    Extracted and separated method init() from JEReplicaDB ctor.
    
    JEReplicaDB.java:
    In ctor, renamed id parameter to serverId + removed queueSize parameter.
    
    JEReplicaDBTest.java:
    Consequence of the change to JEReplicaDB.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9786 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

07 Nov, 2013

1 commit


06 Nov, 2013

6 commits

  • … to diagnose the causes of these events
    
    
    
    Functional tests found a ConcurrentModificationException with r9753 (see at the end of the commit message):
    The code was removing elements from a Collection while it was iterating on the exact same Collection.
    
    
    
    ReplicationBroker.java:
    In inner class LocalEvaluation:
    - Renamed "filteredRSs" to "accepted".
    - Added getAcceptedRSInfos() to allow iterating on them without triggering ConcurrentModificationException.
    In rejectAll*(), called getAcceptedRSInfos().
    In getCSN(), renamed parameters.
    
    Fixed ConcurrentModificationException.
    
    ComputeBestServerTest.java:
    Added a test2ServersUpToDateAnd1EvenMoreUpToDate() to exhibit the ConcurrentModificationException.
    Extracted method newServerState() and used it in all tests.
    In all test methods but one, do not update the ServerState with unused CSNs because they make the tests harder to read. Did not change test3Servers() because I believe it is good to keep one test testing the unneded CSNs do not impact the test results.
    
    
    
    
    java.util.ConcurrentModificationException
    	at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
    	at java.util.HashMap$ValueIterator.next(HashMap.java:822)
    	at org.opends.server.replication.service.ReplicationBroker.rejectAllWithRSOnDifferentVMThanDS(ReplicationBroker.java:1908)
    	at org.opends.server.replication.service.ReplicationBroker.filterServersOnSameHost(ReplicationBroker.java:1878)
    	at org.opends.server.replication.service.ReplicationBroker.computeBestReplicationServer(ReplicationBroker.java:1600)
    	at org.opends.server.replication.service.ReplicationBroker.connectAsDataServer(ReplicationBroker.java:820)
    	at org.opends.server.replication.service.ReplicationBroker.connect(ReplicationBroker.java:691)
    	at org.opends.server.replication.service.ReplicationBroker.start(ReplicationBroker.java:221)
    	at org.opends.server.replication.service.ReplicationDomain.enableService(ReplicationDomain.java:3024)
    	at org.opends.server.replication.service.ReplicationDomain.changeConfig(ReplicationDomain.java:3054)
    	at org.opends.server.replication.plugin.LDAPReplicationDomain.applyConfigurationChange(LDAPReplicationDomain.java:4089)
    	at org.opends.server.replication.plugin.LDAPReplicationDomain.applyConfigurationChange(LDAPReplicationDomain.java:96)
    	at org.opends.server.admin.server.ServerManagedObjectChangeListenerAdaptor.applyConfigurationChange(ServerManagedObjectChangeListenerAdaptor.java:74)
    	at org.opends.server.admin.server.ConfigChangeListenerAdaptor.applyConfigurationChange(ConfigChangeListenerAdaptor.java:342)
    	at org.opends.server.extensions.ConfigFileHandler.replaceEntry(ConfigFileHandler.java:1610)
    	at org.opends.server.workflowelement.localbackend.LocalBackendModifyOperation.processModify(LocalBackendModifyOperation.java:599)
    	at org.opends.server.workflowelement.localbackend.LocalBackendModifyOperation.processLocalModify(LocalBackendModifyOperation.java:299)
    	at org.opends.server.workflowelement.localbackend.LocalBackendWorkflowElement.execute(LocalBackendWorkflowElement.java:690)
    	at org.opends.server.core.WorkflowImpl.execute(WorkflowImpl.java:197)
    	at org.opends.server.core.WorkflowTopologyNode.execute(WorkflowTopologyNode.java:100)
    	at org.opends.server.core.ModifyOperationBasis.run(ModifyOperationBasis.java:451)
    	at org.opends.server.core.SynchronousStrategy.enqueueRequest(SynchronousStrategy.java:49)
    	at org.opends.server.protocols.ldap.LDAPClientConnection.addOperationInProgress(LDAPClientConnection.java:1276)
    	at org.opends.server.protocols.ldap.LDAPClientConnection.processModifyRequest(LDAPClientConnection.java:2268)
    	at org.opends.server.protocols.ldap.LDAPClientConnection.processLDAPMessage(LDAPClientConnection.java:1745)
    	at org.opends.server.protocols.ldap.LDAPRequestHandler.run(LDAPRequestHandler.java:194)
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9784 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • 
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9783 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • CR-2539
    
    * Move resources for EntryGenerator in src/main/resources to have them 
       embedded in opendj-core jar
    * EntryGenerator is not yet able to find these resources wihout setting the path
    
    
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9782 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • CR-2539
    
    * Remove use of schema validation policy as it is useless
    
    
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9781 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • CR-2539
    
    * Rename MakeLDIFEntryReader class to EntryGenerator
    * Rename corresponding test case
    * Rename all related properties in core.properties from xxx_MAKELDIF_xxx to xxx_ENTRY_GENERATOR_xxx
    
    
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9780 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • CR-2539
    
    * Use existing DecodeException class instead of MakeLDIFException class
    ** Simplify try catch clauses in MakeLDIFEntryReader as DecodeException extends IOException
    ** Delete MakeLDIFException class 
    
    * Remove useless throwing of exceptions in methods of MakeLDIFEntryReader 
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9779 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     

05 Nov, 2013

5 commits

  • CR-2539
    
    Initial port of entries generation from opendj server (opendj/trunk/opends/src/server/org/opends/server/tools/makeldif package)
    
    * Behavior related to CLI is not adressed
    * Lots of issues identified in the code review will be adressed in next commits
    
    * Class organization
    ** MakeLDIFEntryReader class is the main entry point to generate entries
    ** TemplateFile class contains all classes related to template parsing and processing excluding tags
    ** TemplateTag class contains all tag-related classes
    ** MakeLDIFEntryReaderTestCase include initial tests from MakeLDIF and some additional tests
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9778 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • In createChangelogEntry(), removed duplicate parameters.
    In createEntryFromMsg(), extracted method addAttribute().
    Changed returnECLControl field to boolean (incorrectly was Boolean)
    Improved formatting.
    Removed comments paraphrasing the code.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9777 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Added complimentary javadocs to explain what these constants mean.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9776 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Renamed variables and changed comments to match the new terminology.
    Added more comments to explain what the code is doing.
    
    ReplicationBroker.java
    Renamed replicationServerInfo => rsInfo.
    
    replication.properties
    Removed space characters at the end of each lines.
    
    RSInfo.java:
    In toString(), removed newline + improved formatting.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9775 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Renamed to collect_all.sh.
    Now also collect config dir.
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9774 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

04 Nov, 2013

5 commits

  • …d (RSUpdater) can spin
    
    
    This change is linked to the misbehaving RSUpdater thread which (wrongly):
    - could be started multiple times
    - would not shutdown willingly when the server is shutting down
    - would try to look for replay operations in the future
    
    
    
    LDAPReplicationDomain.java:
    In now(), ensured now() will always come last with isNewerThan() test.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9769 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • …d (RSUpdater) can spin
    
    
    This change is linked to the misbehaving RSUpdater thread which (wrongly):
    - could be started multiple times
    - would not shutdown willingly when the server is shutting down
    - would try to look for replay operations in the future
    
    
    
    LDAPReplicationDomain.java:
    In buildAndPublishMissingChanges(), only exit the loop when the currentStartCSN is newer than now (instead of the incorrect currentStartCSN + 10s).
    
    HistoricalCsnOrderingTest.java:
    Code cleanup.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9767 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Updated retrieving the source jars.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9766 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • git-svn-id: https://svn.forgerock.org/opendj/trunk@9765 41b1ffd8-f28e-4786-ab96-9950f0a78031
    cjr
     
  • …d (RSUpdater) can spin
    
    
    This change is linked to the misbehaving RSUpdater thread which (wrongly):
    - could be started multiple times
    - would not shutdown willingly when the server is shutting down
    - would try to look for replay operations in the future
    
    
    
    LDAPReplicationDomain.java:
    Added AtomicReference for RSUpdater thread.
    Used DirectoryThread.isShutdownInitiated() for ServerStateFlush and RSUpdater threads.
    In RSUpdater, added shutdown field to pass it down to buildAndPublishMissingChanges() + overrode initiateShutdown() to set the shutdown field to true.
    In shutdown(), called initiateShutdown() for RSUpdater and RSUpdater threads.
    In sessionInitiated(), only start the RSUpdaterThread if it does not already exist.
    In buildAndPublishMissingChanges(), added early exits in case of shutdown.
    Added now() method.
    
    HistoricalCsnOrderingTest.java:
    Consequence of the change to LDAPReplicationDomain.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9764 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

31 Oct, 2013

1 commit

  • In testGetConnectionAsync*(), mocked and verified how the ResultHandler is called.
    Renamed init() to mockConnectionWithInitialHeartbeatResult().
    Removed unneeded comments.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9763 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

30 Oct, 2013

3 commits

  • In runTask(), improved error logging in case of exceptions.
    Little code cleanup.
    
    TaskThread.java
    Logged the task state after the task finished.
    Comments => javadocs.
    
    backend*.properties:
    Updated NOTICE_TASK_FINISHED_414 message. Updated the translations where I could, otherwise removed them (alas).
    
    replication.properties, LDAPReplicationDomain.java:
    Removed useless spaces in error logging
    
    DSInfo.java:
    In toString(), removed newline + improved formatting.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9762 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Added cloneWithReplicationServerId().
    
    LightweightServerHandler.java:
    Simplified ctor by just accepting a DSInfo object.
    Extracted method debugInfo()
    Removed redundant fields.
    
    ReplicationServerHandler.java:
    In processTopoInfoFromRS(), called DSInfo.cloneWithReplicationServerId() + consequence of the change to LightweightServerHandler.
    
    ReplicationDomain.java:
    Small code cleanup.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9761 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • git-svn-id: https://svn.forgerock.org/opendj/trunk@9760 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

29 Oct, 2013

2 commits

  • 
    AssuredReplicationServerTest.java
    Fixed lost configuration lost in r9746.
    
    FakeReplicationDomain.java, ReplicationDomainTest.java:
    Added back configuration accidentally removed in r9746.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9759 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • In createReplicationBroker(), added back configuration accidentally removed in r9746
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9758 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

28 Oct, 2013

5 commits

  • CR-2513
    
    * Add base class LDAPBaseFilter to share code between 
       LDAPServerFilter and LDAPClientFilter classes as suggested 
       in code review.
       
       Allows to share handleRead() method
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9757 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • Complete test case for LDAP reader / writer : all messages supported by LDAPMessageHandler are tested.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9756 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • replication.properties
    Removed the forbidden final dot from i18n message.
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9755 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • Used Assertions to have more descriptive assertion failures.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9754 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • …r in order to diagnose the causes of these events
    
    
    This change records how each replication server is evaluated by computeBestReplicationServer().
    It also logs why a DS switches from one RS to another RS.
    
    
    ReplicationBroker.java:
    Added static classes RSEvaluations and LocalEvaluation.
    Moved keepBest() method to RSEvaluations.
    In filterServersWithAllLocalDSChanges(), reworked the code to be more understandable + Added method getCSN() and rejectAllWithRSIsLaterThanBestRS(), rejectAllWithRSOnDifferentVMThanDS().
    In computeBestServerWhenConnected(), extracted method isServerOverloadingRS().
    
    replication.properties:
    Added tons of messages.
    
    ComputeBestServerTest.java:
    Moved this class to package org.opends.server.replication.service to match the package of the tested code.
    Consequence of the changes to ReplicationBroker.
    Added methods containsOnly(), getDescription(), getEval1() and getEval2().
    Reformatted the code a bit.
    Removed @throws from javadocs.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9753 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

25 Oct, 2013

3 commits

  • git-svn-id: https://svn.forgerock.org/opendj/trunk@9751 41b1ffd8-f28e-4786-ab96-9950f0a78031
    matthew
     
  • Move all classes from org.forgerock.opendj.asn1 package to org.forgerock.opendj.io package in opendj-core
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9750 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • Move non-specific Grizzly code to opendj-core and refactor LDAPReader and LDAPWriter.
    
    * Add new package org.forgerock.opendj.io in opendj-core for all IO related features
    **  org.forgerock.opendj.asn1 classes to be moved in this io package in a future commit 
    
    * Refactor LDAPReader class
    ** use an ASN1Reader as field instead of passing it as arg of each method
    ** rename methods to readXX
    ** add hasMessageAvailable method
    ** move to org.forgerock.opendj.io package in opendj-core
    
    * Refactor LDAPWriter class
    ** use an ASN1Writer as field instead of passing it as arg of each method
    ** rename methods to writeXX
    ** remove implementation of LDAPMessageHandler
    ** move to org.forgerock.opendj.io package in opendj-core
    
    * Adapt GrizzlyLDAPConnection, GrizzlyLDAPConnectionFactory, GrizzlyLDAPListener,
    LDAPClientFilter and LDAPServerFilter classes to changes in LDAPReader and LDAPWriter
    ** caching of LDAPReader per connection
    ** caching of LDAPWriter per thread 
    ** new utility methods in GrizzlyUtils class to handle creation and recycling of LDAPReader and
    LDAPWriter instances
    
    * Refactor LDAPMessageHandler and AbstractLDAPMessageHandler classes
    ** remove user provided P parameter
    ** move to org.forgerock.opendj.ldap.spi package in opendj-core
    
    * Move UnexpectedRequestException, UnexpectedResponseException and UnsupportedMessageexception classes
    to org.forgerock.opendj.ldap.spi package in opendj-core
    
    * Convert TimeoutChecker class to a generic timeout facility which is not tied to connections.
    ** new interface TimeoutEventListener to represent listeners on timeout event
    ** TimeoutChecker send callbacks to listeners instead of connections
    ** move to org.forgerock.opendj.ldap package in opendj-core
    
    * Fix issues with TimeoutChecker and usage in GrizzlyLDAPConnection
    ** prevent negative delays
    ** wakeup of listeners only if timeout is > 0 
    
    * Add test case support for LDAPReader/LDAPWriter in opendj-core and 
    specific implementation of test case in opendj-grizzly
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9749 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     

23 Oct, 2013

2 commits

  • Thanks to Chris Ridd for reviewing the patch I posted to the issue and letting me know by IM.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9747 41b1ffd8-f28e-4786-ab96-9950f0a78031
    mark
     
  • Replaced instance fields with directly storing and using the ReplicationDomainCfg object.
    
    ReplicationBroker.java:
    Replaced instance fields with directly storing and using the ReplicationDomainCfg object.
    Removed start(Set<String>), setChangeTimeHeartbeatInterval() (now useless), and setGroupId() (never used).
    Added getBaseDN(), getReplicationServerUrls(), getGroupId().
    In several methods, used Map.entrySet() instead of Map.keySet() with Map.get(Object)  + renamed replicationServerInfo to rsInfo.
    
    replication.properties:
    Removed now unused message.
    
    
    ReplicationDomain.java:
    Replaced instance fields with directly storing and using the ReplicationDomainCfg object.
    Code cleanups:
    - Removed useless use of "this." for method calls.
    - In processErrorMsg() and processUpdateDone(), collapsed if statements
    - Increased vertical density
    - Extracted method getGenId()
    - In waitForAckIfAssuredEnabled(), used early exit.
    
    
    DomainFakeCfg.java:
    Comments => javadocs
    baseDn => baseDN
    Removed several unused methods.
    Added setWindowSize() + instance member.
    
    *Test*.java
    Used the DomainFakeCfg everywhere + SortedSet instead of Set.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9746 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     

21 Oct, 2013

1 commit


18 Oct, 2013

2 commits

  • This a a part of OPENDJ-175 - Decouple OpenDJ LDAP SDK from Grizzly 
    CR2491
    
    * In GrizzlyLDAPListener and GrizzlyLDAPConnectionFactory classes :
     - Default filter chain used for processing is built from transport instead of building it from scratch.
     - Additional Grizzly filters can be provided by using transport argument passed to constructors (no change implied)
     - Additional Grizzly filters are added between Transport filter and LDAP filter
     
    * Add a new utility class GrizzlyUtils that provides buildFilterChain and addFilterToConnection methods
       Use the utility class to factorize code to add a filter to the filter chain in LDAPServerFilter and GrizzlyLDAPConnection classes
      
    * Add test case for the new GrizzlyUtils class
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9740 41b1ffd8-f28e-4786-ab96-9950f0a78031
    nicolas.capponi@forgerock.com
     
  • * revert back to previous "(objectClass=*)" search filter to avoid interop issues.
    
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9738 41b1ffd8-f28e-4786-ab96-9950f0a78031
    matthew
     

17 Oct, 2013

3 commits

  • 
    ReplicationDbEnv.java, replication.properties:
    In openDatabase(), fixed TODOs for i18n.
    In newErrorMessage(), improved error logging by including the DB name.
    
    JEChangelogDB.java:
    Simplified the code by calling domainToReplicaDBs.remove() from inside shutdownReplicaDBs().
    In shutdownReplicaDBs(), do not bother cleaning up the domainMap since it will be removed from domainToReplicaDBs.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9737 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • …e directory server's broker was not connected to a replication domain, hence it was impossible to add the changes to the changelogDB and then find them when querying the changelogDB.
    
    
    GenerationIdTest.java:
    Renamed assertConnectedToReplicationDomain() to waitConnectionToReplicationDomain().
    
    LDAPReplicationDomain.java:
    Removed uses of StringBuilder, relied on *Operation.toString() to do it for us.
    
    ReplicationBroker.java:
    Improved toString().
    
    ReplicationTestCase.java:
    Removed a wrong comment.
    Used Assertions.assertThat().
    
    ReplicationServerTest.java:
    Removed a useless call to DirectoryServer.getSynchronizationProviders().
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9736 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac
     
  • … that it is failing because the server is not connected to a replication domain, hence it is impossible to add the changes to the changelog.
    
    GenerationIdTest.java:
    Added assertConnectedToReplicationDomain() + called it.
    Fixed a few logged strings.
    
    git-svn-id: https://svn.forgerock.org/opendj/trunk@9735 41b1ffd8-f28e-4786-ab96-9950f0a78031
    JnRouvignac