Help with Account disabled test and Open LDAP(Novell) about redmine_ldap_sync HOT 84 CLOSED

The plugin would only allow to use one of those attributes.

You can set:

Account control flags: loginDisabled
Account disabled test: flags == 'true'

from redmine_ldap_sync.

i did exactly what you said, but my accounts still don't get unlocked, altough the flag is false. Is it one way?

from redmine_ldap_sync.

Yes, it is one way on purpose.
The idea is that one might need to lock one account on redmine but not on LDAP.

If it comes handy to you that the plugin also unlocks the users, I could try to make it configurable.

from redmine_ldap_sync.

Normally the user shouldN't be able to login if ldap has account disbled, because ebery login queries ldap for auth.
But however is such feature exists it is useful to be able to revert. Imagin there is any bug with this switching and you have to manaully (or by sql) unlock 1000 users. It would be useful to have a switch for the rake task to be able to also unlock again.

Tx in advance.

from redmine_ldap_sync.

Done, it should work as suggested.

from redmine_ldap_sync.

Please reopen this issue. My users still get locked although the account flag "loginDisabled" is set tor FALSE in ldap and i configured "Account disabled test" set to "flags == 'TRUE'"

Could it be a bug and the cause is if flagname in config does not match ldap flag? The names are indetically but not case sensitive.

ldap: "loginDisabled"
plugins config": "logindisabled"

I can't get the "D" Uppercase, the plugin always chnages all to lowercase when saving the form.

from redmine_ldap_sync.

Regarding the attributes, LDAP is case insensitive.

You can try the following
flags.to_s.downcase == 'true'

I'm not sure what is the type of the flags on your case.
If it is a boolean this solution might work.
If it is a string, it will do no harm.

from redmine_ldap_sync.

I didn't meant the flag value case senstive might the problem but the name of the flag, which plugin reformats always all downcase and ldap flags names in upper "D" in loginDisabled.

from redmine_ldap_sync.

flags.to_s.downcase == 'true' doesn't help by the way. User gets locked again. Is there any debug i can see how the expression is evaluated to trace the problem.

I think the loginDisbled is of Type boolean, bevause i get a drop down with TRUE or FALSE in my ldap client.

from redmine_ldap_sync.

I believe the problem is on value of the flags variable.
If ldap_sync doesn't find the attribute, the method account_disabled? returns false.
The only way for it to return true is for the account_disabled_test to return true.

You can try to use the following value:

Rails.logger.error('!!!Flags: ' + flags.inspect); flags.to_s.downcase == 'true'

And the look at the logs to see what's the value of flags.

If Rails is not captured by the lambda, it might throw an error and do nothing.

from redmine_ldap_sync.

Sory, where shall i put this log message exactly? Need some more hints..

from redmine_ldap_sync.

On the plugin's configuration page, set it as the value of the "Account disabled test".

Account disabled test = Rails.logger.error('!!!Flags: ' + flags.inspect); flags.to_s.downcase == 'true'

from redmine_ldap_sync.

If this is not enough place some puts on auth_source_ldap_patch.rb and try the rake task again:

      find_all_users(ldap_con, [:login, :account_flags]) do |entry|
        if account_disabled?(entry[:account_flags])
          puts "!!!User: #{entry.inspect} disabled"
          users[:disabled] << entry[:login]
        else
          puts "!!!User: #{entry.inspect} enabled"
          users[:enabled] << entry[:login]
        end
      end
    end

from redmine_ldap_sync.

rake redmine:plugins:redmine_ldap_sync:sync_users RAILS_ENV=production

Synchronizing AuthSource ldap...
-- Locked user '00001276'
-- Updating user '00000960'...
-> 34 groups added

--- LOG ---
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "TRUE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "TRUE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"
!!!Flags: "FALSE"

from redmine_ldap_sync.

I added new debug code into the class also, but only puts the disbled users and what is very strange is that the users getting disabled are not also put as regular log message, there only user 00001276 is logged as "Locked" and one as updated . I don't undertsand this. I thought Locking=disbled, am i worng?
The user marked as Updated gets updated all the time, even if i call this rake rask in an endless loop, although the user doesn't change, thats strange also.

user@dev-infomine:~/DevMine_WK/infomine-2.2> rake redmine:plugins:redmine_ldap_sync:sync_users RAILS_ENV=production

Synchronizing AuthSource ldap...
!!!User: {:user_memberid=>"cn=00136502,ou=SPCxx,ou=S,o=D", :account_flags=>"TRUE", :login=>"00136502", :groupname=>"00136502"} disabled
!!!User: {:user_memberid=>"cn=21001138,ou=Testuser,ou=S,o=D", :account_flags=>"TRUE", :login=>"21001138", :groupname=>"21001138"} disabled
-- Locked user '00001276'
-- Updating user '00000960'...
-> 34 groups added
Synchronizing AuthSource replic...
user@dev-infomine:~/DevMine_WK/infomine-2.2>

from redmine_ldap_sync.

Ok, so the correct expression for the test is flags == 'TRUE'. (the value is a string with uppercase letters)

If it is locking the users regardless of the expression there must be a bug somewhere else.
On the plugin configuration, do you have something on "Users must be member of"?

from redmine_ldap_sync.

-8dc61b770b78.jpg)

from redmine_ldap_sync.

The default behaviour is that for every found user it shows either nothing or one of the 3 messages messages on the following situations:

• "-- Locked... ": If the user was unlocked and is now being locked (due to the account disabled test)
• "-- Updating ... ": If the user was neither locked and is not disabled on ldap and means that it will now try to update the groups and the user fields (mail and names).
• "-- Creating ...": If the user is not locked and the is new on the redmine's user database
• <Nothing is shown>: If the user is locked on redmine, regardless is the user is disabled or not on ldap.

Try to run the rake task as the following to see if it unlocks the users:
rake redmine:plugins:redmine_ldap_sync:sync_users ACTIVATE_USERS=1 RAILS_ENV=production

User '00000960' shows up always because it is not being locked.
Most probably you have only seen the "-> 34 groups added" message once.

from redmine_ldap_sync.

I set disbale rule to "Rails.logger.error('!!!Flags: ' + flags.inspect); flags.to_s.downcase == 'TRUE'"

and then excecuted "rake redmine:plugins:redmine_ldap_sync:sync_users ACTIVATE_USERS=1 RAILS_ENV=production"

user@dev-infomine:~/DevMine_WK/infomine-2.2> rake redmine:plugins:redmine_ldap_sync:sync_users ACTIVATE_USERS=1 RAILS_ENV=production

Synchronizing AuthSource ldap...
-- Locked user '00001276'
-- Updating user '00000960'...
-> 34 groups added
Synchronizing AuthSource replic...
user@dev-infomine:~/DevMine_WK/infomine-2.2>

Does not work. ;/

from redmine_ldap_sync.

Something is very strange here. All the time he is doing the same thing Locking user 1276, updating user 960 and adding 34 groups ..
Lookms like running in dry mode all the sme - lol

from redmine_ldap_sync.

You have a quite strange redmine environment... it's like it was rolling back the changes every time.

You are using the latest version of the branch master, aren't you?

from redmine_ldap_sync.

If you have disabled the "Create groups" option it's possible for you get that "-> 34 groups" all the time.
The reason is that he finds 34 groups to wich the user doesn't belong to but since it cannot create them it ends not adding the user to those groups.

from redmine_ldap_sync.

User '00001276' might be getting locked every time (without success) if it is invalid. For example, if you added a new required custom field or it is missing the firstname/lastname/email.

You can confirm this by trying to edit the user '00001276' on redmine.

(Later I'll try to fix this two problems)

from redmine_ldap_sync.

Just make sure of the following:

• You're using the latest master version
  (or at least you have the following changes ea33eb2)
• You have account disabled test = flags = 'TRUE'

And run rake redmine:plugins:redmine_ldap_sync:sync_users ACTIVATE_USERS=1 RAILS_ENV=production

You plugin configuration looks good.

from redmine_ldap_sync.

Maybe the group are not added because i didn't set the flag "create groups" what is ok, because i don't want to create groups only to sync group membership in redmine like in ldap for in redmine existing groups and existing users. The log message is misleading, and shall not appear if the flag for "create groups is not set". In my case if group membership is synced on redmine site, it shall name is "Updates group membership" if not this is part of the "update user" process.
Now i use
https://github.com/thorin/redmine_ldap_sync.git 'master' on redmine 2.2.2 stable
The only difference is that i use another net-ldap gem
gem "net-ldap-1", "~> 0.4.0" because of #49. I juts noticed about the secuity issue and fall back to standard gem, hopefully i don't run into the utf8 problem again.

from redmine_ldap_sync.

So, both with master, flags = 'TRUE' and ACTIVATE_USERS=1 it doesn't unlock the users, right?

Sorry to insist, but after so many tests it's not clear to me if you already tried with this settings.

from redmine_ldap_sync.

Ok, i removed and gem uninstalled net-ldap-1 and run into the utf8 bug again. I fixed the standard 0.31 gem manually with https://github.com/ruby-ldap/ruby-net-ldap/pull/44/files and got rid of this bug.

Now I have exactly the master (just did a clean clone), "flags = 'TRUE'" and called "rake redmine:plugins:redmine_ldap_sync:sync_users ACTIVATE_USERS=1 RAILS_ENV=production"

My user for example, which is "0001276" and many more other users get locked again, and i don't now why.

The ldap fields for this user exists in ldap server and are well set:

cn (uid), mail, givenName (first name), sn (last name) and loginDisabled

User don't get unlocked in redmine if already locked and gets locked if not locked after i execute
"rake redmine:plugins:redmine_ldap_sync:sync_users ACTIVATE_USERS=1 RAILS_ENV=production"

Why does for example the user 00001276 gets locked if none of the "Puts" before prints out this user id ?????

See debug bcode:

find_all_users(ldap_con, [:login, :account_flags]) do |entry|
if account_disabled?(entry[:account_flags])
puts "!!!User: #{entry.inspect} disabled"
users[:disabled] << entry[:login]

(def ldap users) !=

My summary
1.) The cause for above fact is not account_disabled flag

from redmine_ldap_sync.

You have some good questions. :)

The answer as to be where we are not looking:

 users_on_local    = self.users.active.map {|u| u.login.downcase }
 users_on_ldap     = users.values.sum.map(&:downcase)
 users[:disabled]  += users_on_local - users_on_ldap

This disables all the users that belongs to the current authentication source if they where not found on ldap.

Do you have two authentication sources with colliding users?
[review: this question doesn't make much sense. When locking the users, it only locks the ones that belong to the current auth source]

from redmine_ldap_sync.

To help understand what's happening you can add some puts after those lines:

 users_on_local    = self.users.active.map {|u| u.login.downcase }
 users_on_ldap     = users.values.sum.map(&:downcase)
 users[:disabled]  += users_on_local - users_on_ldap
 puts "local users = #{users_on_local.inspect}"
 puts "ldap users = #{users_on_ldap.inspect}"
 puts "users = #{users.inspect}"

It will show the final list of enabled and disabled users, before it starts locking them.

from redmine_ldap_sync.

I have two ldap sources defined an use only one. ldap_sync config also has only one activated, maybe you shall check this flag and don't include non activated auth sources in any data retrieval.
Beyond that i don't have any more auth sources then internal redmine base, what makes 3.

I removed the second ldap source, but error remains.

from redmine_ldap_sync.

select distinct auth_source_id from users; returns 0 and 1
select distinct status from users; returns 0 and 1 and 3

select status, count(status) from users group by status (What is this value?)
0 1
1 60
3 247

from redmine_ldap_sync.

active, locked, registered - what is what, stgh todo with problems?

from redmine_ldap_sync.

STATUS_ANONYMOUS = 0
STATUS_ACTIVE = 1
STATUS_LOCKED = 3

from redmine_ldap_sync.

Ok, all ldap user got locked by task, only redmine internal user are still active

from redmine_ldap_sync.

The plugin takes into account the activated flag.
If an auth_source isn't active it doesn't touch it.
All the user locking, unlocking, update is done only on users associated to active auth_sources.

Can you include those puts I sent you before?

 users_on_local    = self.users.active.map {|u| u.login.downcase }
 users_on_ldap     = users.values.sum.map(&:downcase)
 users[:disabled]  += users_on_local - users_on_ldap
 puts "local users = #{users_on_local.inspect}"
 puts "ldap users = #{users_on_ldap.inspect}"
 puts "users = #{users.inspect}"

from redmine_ldap_sync.

How do you have you authentication source configured? Have you made any recent changes?

Have you added an filter (user_filter)? Have you changed the base dn?

from redmine_ldap_sync.

from redmine_ldap_sync.

I inserted the debug code:

puts "local users = #{users_on_local.inspect}"
puts "ldap users = #{users_on_ldap.inspect}"
puts "users = #{users.inspect}"

For the example user 1276 which gets locked it look like this

local users = ["00001276",....]
ldap users = not inlcuded <<---------------- This is strange!
users = {:enabled=>#<Set: {"00001276",....} <<---------------- so its enabled, why its locked then???

from redmine_ldap_sync.

so why its not in the ldap list , i inserted debug code again i found out that user 1276 is not listed in users var after the find_all_users() method

from redmine_ldap_sync.

Was it both on :enabled and on :disabled?
There's an issue I wasn't able to solve that might be related: #27

from redmine_ldap_sync.

the find_all_users() method does not return this user, so neither activated or deactivated

from redmine_ldap_sync.

But i don't use this field, "Must be member of" see my above screenshots from plugins settings.

#56 (comment)

from redmine_ldap_sync.

I've seen it... but he also add a problem with the locking of users that were enabled.

from redmine_ldap_sync.

What do you mean with ldap users = not included
Was that the exact message?

from redmine_ldap_sync.

I've done one more change to see if it solves it.

If not, I'll would give it some time and see if I can come up with an explanation.

Just one question. Does the users only get locked when you have account control flags and account disabled test?

Thank you for your patience

from redmine_ldap_sync.

What do you mean with ldap users = not included

No, i just want to say that the user id value is not in the set/array

I've done one more change to see if it solves it.

Sadly, its not solved by the change.

Just one question. Does the users only get locked when you have account control flags and account disabled test?

I don't understand, not having both set doesn't make any sense to me. How shall the plugin now the name of the flag if not set and how does it check the flaf if not se the condition? Are there defaults? In my case none of any defaults would match, does it? So both has to be set, or nothing, if i won't the disability to be tested.

from redmine_ldap_sync.

You have to check find_all_users() , stgh filters out or does not query correctly, so that user's don't get reqtrieved completly.

from redmine_ldap_sync.

Maybe the user class? The attrib name is objectClass and user has many values, e.g organizationalPerson what i set in plugins settings.
Maybe the filter only queries one values or stgh like this?

from redmine_ldap_sync.

My question was, assuming you were using the plugin before, if the users only started to get locked once you configured the account control flags and the account disabled test.
Without those configurations the plugin is designed to only lock users that were removed from ldap.

So far, the information I have been able to gather is a bit confusing.
If the user 00001276 is on users_on_local and is not on users_on_ldap then it shouldn't be on users[:enabled].

# get all active users on current auth source
users_on_local    = self.users.active.map {|u| u.login.downcase }
# get all users both on :enabled and :disabled
users_on_ldap     = users.values.sum.map(&:downcase)
# removes from the local users, those that were found on ldap and add them to the disabled users
users[:disabled]  += users_on_local - users_on_ldap

According to the information that you have given me this code is not doing what is expected.
The user was on users[:enabled] but it wasn't on users_on_ldap
But later it shows up on users[:disabled] because the message "-- Locked user '00001276'" is printed.

You can try with any of the objectClasses.
But if all your users belong to the objectClass organizationalPerson, that setting is fine.

from redmine_ldap_sync.

I've fixed the two previous problems and added some trace messages to have more details of what's happening underneath the synchronization.

from redmine_ldap_sync.

Its buggy:
rake redmine:plugins:redmine_ldap_sync:sync_users ACTIVATE_USERS=1 RAILS_ENV=production --trace

** Invoke redmine:plugins:redmine_ldap_sync:sync_users (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute redmine:plugins:redmine_ldap_sync:sync_users
Synchronizing AuthSource ldap...
rake aborted!
undefined method downcase' for nil:NilClass /home/sidfunktion/.rvm/rubies/ruby-1.9.3-p327/lib/ruby/1.9.1/set.rb:222:inblock in each'
/home/sidfunktion/.rvm/rubies/ruby-1.9.3-p327/lib/ruby/1.9.1/set.rb:222:in each_key' /home/sidfunktion/.rvm/rubies/ruby-1.9.3-p327/lib/ruby/1.9.1/set.rb:222:ineach'
/home/sidfunktion/DevMine_WK/infomine-2.2/plugins/redmine_ldap_sync/lib/redmine_ldap_sync/redmine_ext/auth_source_ldap_patch.rb:152:in map' /home/sidfunktion/DevMine_WK/infomine-2.2/plugins/redmine_ldap_sync/lib/redmine_ldap_sync/redmine_ext/auth_source_ldap_patch.rb:152:inldap_users'
/home/sidfunktion/DevMine_WK/infomine-2.2/plugins/redmine_ldap_sync/lib/redmine_ldap_sync/redmine_ext/auth_source_ldap_patch.rb:44:in sync_users' /home/sidfunktion/DevMine_WK/infomine-2.2/plugins/redmine_ldap_sync/lib/tasks/sync_users.rake:23:inblock (5 levels) in <top (required)>'
/home/sidfunktion/DevMine_WK/infomine-2.2/plugins/redmine_ldap_sync/lib/tasks/sync_users.rake:21:in each' /home/sidfunktion/DevMine_WK/infomine-2.2/plugins/redmine_ldap_sync/lib/tasks/sync_users.rake:21:inblock (4 levels) in <top (required)>'
/home/sidfunktion/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/task.rb:228:in call' /home/sidfunktion/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/task.rb:228:inblock in execute'
/home/sidfunktion/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/task.rb:223:in each' /home/sidfunktion/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/task.rb:223:inexecute'
/home/sidfunktion/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/task.rb:166:in block in invoke_with_call_chain' /home/sidfunktion/.rvm/rubies/ruby-1.9.3-p327/lib/ruby/1.9.1/monitor.rb:211:inmon_synchronize'
/home/sidfunktion/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/task.rb:159:in invoke_with_call_chain' /home/sidfunktion/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/task.rb:152:ininvoke'
/home/sidfunktion/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/application.rb:143:in invoke_task' /home/sidfunktion/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/application.rb:101:inblock (2 levels) in top_level'
/home/sidfunktion/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/application.rb:101:in each' /home/sidfunktion/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/application.rb:101:inblock in top_level'
/home/sidfunktion/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/application.rb:110:in run_with_threads' /home/sidfunktion/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/application.rb:95:intop_level'
/home/sidfunktion/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/application.rb:73:in block in run' /home/sidfunktion/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/application.rb:160:instandard_exception_handling'
/home/sidfunktion/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/application.rb:70:in run' /home/sidfunktion/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/bin/rake:33:in<top (required)>'
/home/sidfunktion/.rvm/gems/ruby-1.9.3-p327/bin/rake:23:in load' /home/sidfunktion/.rvm/gems/ruby-1.9.3-p327/bin/rake:23:in

from redmine_ldap_sync.

With or without account_flags configured?

That error means some of the users don't have 'cn'.

from redmine_ldap_sync.

With account_flags set

flags = 'TRUE'

What yo you mean with "some of the users" ? The test user 1276 we spoke all the timei yet checked and it has a cn.

from redmine_ldap_sync.

You setup is behaving completely random... :(
I'll do something to prevent the nilClass error.
But that error shouldn't be happening at all.

The list of users shouldn't contain 'nil's at all. So that downcase on a nil error means that it is not being able to retrieve the users correctly from ldap.

from redmine_ldap_sync.

if you place a puts changes.inspect before line 152 you'll see that the sets contains nils...
It means that find_all_users is generating entries without the user login.

Yesterday we didn't had that error and I haven't changed that part of the code... :(

from redmine_ldap_sync.

Might be because of the net-ldap patch. Try disabling it on init.rb.

from redmine_ldap_sync.

Wait, i made a clean clone and removbed myslf fixed net-ldap patch in favour of yours.
Now the task runs, but user 1276 i still locked

rake redmine:plugins:redmine_ldap_sync:sync_users ACTIVATE_USERS=1 RAILS_ENV=production --trace

** Invoke redmine:plugins:redmine_ldap_sync:sync_users (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute redmine:plugins:redmine_ldap_sync:sync_users
Synchronizing AuthSource dsv-ldap...
-- Found 320 users active, 1417 locked and 0 deleted on ldap
.....
-- Locked active user '00001276'
.....
Above entry is the only witch macthes consoel output for "1276" string match

I migrated to redmine 2.2.3 in the meantime

from redmine_ldap_sync.

For undertanding you first go through a look to lock users, then unlock users.
That would explain the hundreds of
-- Not locking locked user 'userid'

strings in the first place

from redmine_ldap_sync.

My guess is that this is an encoding problem. I would like to try to recreate the problem on my lap. I'll need an example of an user's first name, last name, and login.

The most important thing is to know if any of those fields have non ASCII characters.

I will also need to know what is the encoding that is being used by ruby and by novell edirectory.
For that I'll try to give you some directions later.

from redmine_ldap_sync.

Can you confirm that you are using ruby 1.9?
I believe the problems with net_ldap and the encoding errors started showing up with this version of ruby.

--- For ruby 1.9 you can do the following to get the encodings:

redmine# rails console RAILS_ENV=production
Encoding.find('internal')
Encoding.find('external')
Encoding.find('locale')
Encoding.find('filesystem')
Encoding.locale_charmap

-- I'll also need the encoding of the database:
For Mysql: show variables like "%char%";
For postgresql (on the redmine's production db): SHOW SERVER_ENCODING;

-- I believe that Novell eDirectory only works with UTF-8 internally and that is not possible to change
Can you confirm it?

from redmine_ldap_sync.

Can you confirm that you are using ruby 1.9?

ruby 1.9.3p327 (2012-11-10 revision 37606) [i686-linux]

Loading development environment (Rails 3.2.12)
irb: warn: can't alias help from irb_help.
1.9.3-p327 :001 > Encoding.find('internal')
=> #Encoding:UTF-8
1.9.3-p327 :002 > Encoding.find('external')
=> #Encoding:UTF-8
1.9.3-p327 :003 > Encoding.find('locale')
=> #Encoding:UTF-8
1.9.3-p327 :004 > Encoding.find('filesystem')
=> #Encoding:UTF-8
1.9.3-p327 :005 > Encoding.locale_charmap
=> "UTF-8"
1.9.3-p327 :006 >

show variables like "%char%"
'character_set_client' 'utf8'
'character_set_connection' 'utf8'
'character_set_database' 'utf8'
'character_set_filesystem' 'binary'
'character_set_results' ''
'character_set_server' 'utf8'
'character_set_system' 'utf8'
'character_sets_dir' '/usr/share/mysql/charsets/'

I believe that Novell eDirectory only works with UTF-8 internally and that is not possible to change

Sorry, i am just no idea, Our admin also don't know how to find out, bad ins't it?

from redmine_ldap_sync.

Ok i finally found out the ldap server is UTF-8. (LC_CTYPE=en_US.UTF-8.)

from redmine_ldap_sync.

ok, All I need now is an example of a user that fails to synchronize.

You told me it fails to enable your user.
Does your user have non-ascii letters on the login, firstname or lastname?

from redmine_ldap_sync.

After giving me an example you can try changing the file lib / redmine_ldap_sync / core_ext / string_patch.rb to the following:

module RedmineLdapSync::CoreExt::StringPatch
  def raw_utf8_encoded
    return self

    if self.respond_to?(:encode)
      # Strings should be UTF-8 encoded according to LDAP.
      # However, the BER code is not necessarily valid UTF-8
      # self.encode('UTF-8', invalid: :replace, undef: :replace, replace: '' ).force_encoding('ASCII-8BIT')
      begin
        self.encode('UTF-8').force_encoding('ASCII-8BIT')
      rescue Encoding::UndefinedConversionError
        self
      end
    else
      self
    end
  end
  private :raw_utf8_encoded
end

I'm starting to suspect that this convertion might not be needed.
But, if it doesn't work, please revert it back to the way it was.

from redmine_ldap_sync.

You told me it fails to enable your user.

Yes, thats correct.

Does your user have non-ascii letters on the login, firstname or lastname?

Nope, he doesn't

• id is 0-9
• all names are a-Z
• no Umlaute and no german special jars, like "ß"

With ur above patch i get the encoding error again ;/

rake redmine:plugins:redmine_ldap_sync:sync_users ACTIVATE_USERS=1 RAILS_ENV=production --trace

** Invoke redmine:plugins:redmine_ldap_sync:sync_users (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute redmine:plugins:redmine_ldap_sync:sync_users
Synchronizing AuthSource dsv-ldap...
rake aborted!
incompatible character encodings: ASCII-8BIT and UTF-8
/home/user/.rvm/gems/ruby-1.9.3-p327/gems/net-ldap-0.3.1/lib/net/ber/core_ext/array.rb:62:in to_ber_seq_internal' /home/user/.rvm/gems/ruby-1.9.3-p327/gems/net-ldap-0.3.1/lib/net/ber/core_ext/array.rb:54:into_ber_contextspecific'
/home/user/.rvm/gems/ruby-1.9.3-p327/gems/net-ldap-0.3.1/lib/net/ldap.rb:1398:in block in search' /home/user/.rvm/gems/ruby-1.9.3-p327/gems/net-ldap-0.3.1/lib/net/ldap.rb:1367:inloop'
/home/user/.rvm/gems/ruby-1.9.3-p327/gems/net-ldap-0.3.1/lib/net/ldap.rb:1367:in search' /home/user/.rvm/gems/ruby-1.9.3-p327/gems/net-ldap-0.3.1/lib/net/ldap.rb:637:insearch'
/home/user/DevMine_WK/infomine-2.2/plugins/redmine_ldap_sync/lib/redmine_ldap_sync/redmine_ext/auth_source_ldap_patch.rb:297:in ldap_search' /home/user/DevMine_WK/infomine-2.2/plugins/redmine_ldap_sync/lib/redmine_ldap_sync/redmine_ext/auth_source_ldap_patch.rb:289:infind_all_users'
/home/user/DevMine_WK/infomine-2.2/plugins/redmine_ldap_sync/lib/redmine_ldap_sync/redmine_ext/auth_source_ldap_patch.rb:142:in ldap_users' /home/user/DevMine_WK/infomine-2.2/plugins/redmine_ldap_sync/lib/redmine_ldap_sync/redmine_ext/auth_source_ldap_patch.rb:44:insync_users'
/home/user/DevMine_WK/infomine-2.2/plugins/redmine_ldap_sync/lib/tasks/sync_users.rake:23:in block (5 levels) in <top (required)>' /home/user/DevMine_WK/infomine-2.2/plugins/redmine_ldap_sync/lib/tasks/sync_users.rake:21:ineach'
/home/user/DevMine_WK/infomine-2.2/plugins/redmine_ldap_sync/lib/tasks/sync_users.rake:21:in block (4 levels) in <top (required)>' /home/user/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/task.rb:228:incall'
/home/user/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/task.rb:228:in block in execute' /home/user/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/task.rb:223:ineach'
/home/user/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/task.rb:223:in execute' /home/user/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/task.rb:166:inblock in invoke_with_call_chain'
/home/user/.rvm/rubies/ruby-1.9.3-p327/lib/ruby/1.9.1/monitor.rb:211:in mon_synchronize' /home/user/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/task.rb:159:ininvoke_with_call_chain'
/home/user/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/task.rb:152:in invoke' /home/user/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/application.rb:143:ininvoke_task'
/home/user/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/application.rb:101:in block (2 levels) in top_level' /home/user/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/application.rb:101:ineach'
/home/user/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/application.rb:101:in block in top_level' /home/user/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/application.rb:110:inrun_with_threads'
/home/user/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/application.rb:95:in top_level' /home/user/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/application.rb:73:inblock in run'
/home/user/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/application.rb:160:in standard_exception_handling' /home/user/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/lib/rake/application.rb:70:inrun'
/home/user/.rvm/gems/ruby-1.9.3-p327/gems/rake-10.0.3/bin/rake:33:in <top (required)>' /home/user/.rvm/gems/ruby-1.9.3-p327/bin/rake:23:inload'
/home/user/.rvm/gems/ruby-1.9.3-p327/bin/rake:23:in `

from redmine_ldap_sync.

Maybe the problem are other ldap attribs, like groups or description. There i have Umlaute and special Chars which are know not to be supported well from some ldap clients, although shall be asci. See http://subversion.open.collab.net/ds/viewMessage.do?dsForumId=3&dsMessageId=467831

We had problem with the "§" char in password for examle, however that is another client setup with http browser frontend and apache moddav passthrough for ldap auth.

How shall i support you best to find the attrib/chars maybe making issue? Can a charsez problem somwhere in attrib stream break out and affect other attrib, the wehole record for the user resulting in getting locked? However the example user making troubles jas no password with speical chars either. Password also is a-Z, no Umlaute and no german special jars, like "ß" or "§"

from redmine_ldap_sync.

That is an option, but I don't believe it is the case.
A second option is that the conversion between enconding is messing up the search which ends in the user not being found on LDAP.
If the user is not found, it ends up being locked.

I'll give you some other code to test.

from redmine_ldap_sync.

Please try the following code:

module RedmineLdapSync::CoreExt::StringPatch
  def raw_utf8_encoded
    if self.respond_to?(:encode)
      # Strings should be UTF-8 encoded according to LDAP.
      # However, the BER code is not necessarily valid UTF-8
      # self.encode('UTF-8', invalid: :replace, undef: :replace, replace: '' ).force_encoding('ASCII-8BIT')
      if (self.encoding.name == 'UTF-8')
        self.force_encoding('ASCII-8BIT')
      else
        self
      end
    else
      self
    end
  end
  private :raw_utf8_encoded
end

This should prevent raw_utf8_encoded to be applied to control strings.
Those strings should already be in ASCII-8BIT and we don't want to mess them by converting them to UTF-8.

from redmine_ldap_sync.

Nothing changed, user get locked again / not unlocked.

from redmine_ldap_sync.

Ok, I'll keep trying to reproduce the error on my ldap.
I've been moving half blind and right now I have no clue of what might be causing the problem.

By default, redmine only allows ascii characters on the login. (/\A[a-z0-9_\-@\.]*\z/i)
Do you have any modification or plugin that could have changed this default?

from redmine_ldap_sync.

By default, redmine only allows ascii characters on the login. (/\A[a-z0-9_-@.]*\z/i)

The user don't get registed from redmine and pushed into ldap, but created in ldap and synced into redmine.
The user data is read only from redmine site and data i created with some (unknown) other ldap client via enterprise paper based process and manual work.

Do you have any modification or plugin that could have changed this default?

The data is not created via redmine ,s o it shouldn't maptter, but i don't know ones of the plugin i use to do stgh in this direction.

from redmine_ldap_sync.

I was stating it because when synchronizing the users are created on the redmine's DB.
If somehow this filter was changed it would allow those users with non-ascii logins to be created on the database.

If this was not changed redmine_ldap_sync would give an error because redmine wouldn't allow such a user to be created on the DB.

My concern is just to know if I should or not test users with non-ascii characters on the login.
If no changes were made to the filter it makes no sense for me to test it.

from redmine_ldap_sync.

Hm i still don't understand. The user are manually created either via register or Admin>User>New from Redmine Admin.
So you think a plugin could patch the view or controller and entered nonascii chars instead redmine core class which don't.
Why redmine doesn't allow any utf8 for users data if mysql allows utf8. Why don't ur plugin uses utf8 and tries to convert from here to there.
If such problems would exist, why can i log in regular in redmine and get authenticated correctly without problems?
The decision "get lcoked" or get "unlocked" shall evaluate only one ldap attrib to true or false, and then set the model flag according. Maybe we have to check there again. That the only thinh i see so far doesn't work. Why could so much other code now could be resonsible for this issue?

from redmine_ldap_sync.

As I said, my question is only to help me know what I should test.
I'm not pushing the responsibility to another plugin.

So you think a plugin could patch the view or controller and entered nonascii chars instead redmine core class which don't.
No, it's neither a view nor a controller. It is the User model.

Every time a user is saved on the database, it as to go through the validations on the user model.
One of the validations is that the login can only have ascii characters (the regex /\A[a-z0-9_-@.]*\z/i).

Both on login (on-the-fly), or by synchronization, it as to cross the User model to save the user on the local database.
If the validation fails an error is raised and it stops the user from being created.

Yes, it is possible to patch this validation on a plugin if you want to be less restrictive on the logins that you want to allow.

From the tests we've made, I believe there is no problems with evaluation but with finding the users on the ldap.
As you told me the users_on_ldap variable was missing some users.

If a user is missing on ldap it also ends getting locked.

If such problems would exist, why can i log in regular in redmine and get authenticated correctly without problems?

I don't know. I have to do tests. The plugin searches for users using ldap calls different from those used by redmine.

from redmine_ldap_sync.

Do you query for all users of given class and in given BASE DN?
Is thats the code which return not all users, than here is the error. So i could maybe aks my admin to live debug my ldap query on server side and find out why user onjects are no returned.
Or do you think the plugin can no well deserilaize the retunred ldap query result?

from redmine_ldap_sync.

I believe there is some problem deserializing the query result because we have seen that some of the users are missing the cn attribute.

But if will help if the ldap admin could tell us what's query that is reaching the LDAP, how many users it is returning and if the locked users are in it.

The plugin does a query for the attributes cn and loginDisabled of the entries on base dn (O=D...) with the given objectClass (organizationalPerson) .

from redmine_ldap_sync.

For debugging on server side it would help only this problemtic query will be send. Are there other queries the script does, and how could i reduce bit only do the query do test.

from redmine_ldap_sync.

For the record i must really thank you for this strong support and interest to find the issue. I really hope it doesn't turn out to be a false positive because me overseeing stgh. nothing todo with the plugin code.
I am really thank full because the i need the plugin to get rid of the manuals user managment tasks in our growing user base. Thakns that you such fullblooded opensource enthusiast. ;)

from redmine_ldap_sync.

Yes, there are other queries.
If you want to reduce the number of queries sent by the plugin you can introduce an error at the end of the ldap_users method.
For example, by adding nil to the end of the method and then execute the rake synchronization task.

          def ldap_users
            return @ldap_users if @ldap_users

            ldap_con = initialize_ldap_con(self.account, self.account_password)
            changes = {:enabled => Set.new, :disabled => Set.new}

            if settings[:account_flags].blank?
              changes[:enabled] = find_all_users(ldap_con, [:login])
            else
              find_all_users(ldap_con, [:login, :account_flags]) do |entry|
                if account_disabled?(entry[:account_flags])
                  changes[:disabled] << entry[:login] if entry[:login]
                else
                  changes[:enabled] << entry[:login] if entry[:login]
                end
              end
            end

            users_on_local  = self.users.active.map {|u| u.login.downcase }
            users_on_ldap   = changes.values.sum.map(&:downcase)
            deleted_users   = users_on_local - users_on_ldap
            changes[:disabled]  += deleted_users

            msg = "-- Found #{changes[:enabled].size} users active"
            msg << ", #{changes[:disabled].size - deleted_users.size} locked"
            msg << " and #{deleted_users.size} deleted on ldap"
            puts msg

            @ldap_users = changes
            nil # This will cause an undefined method '[]' for nil:NilClass Error
          end

Thank you for your thanks. I really appreciate it.

PS: I hope that with the next version of the plugin it becomes easier to configure and test all this ldap settings.

from redmine_ldap_sync.

I nearly found the cause. IT's the method account_disabled which returns 'TRUE' for '00001276's 'account_flags=FALSE' and the configured account deativation condition "flags = 'TRUE'"

Debug Code:

def ldap_users
return @ldap_users if @ldap_users

        ldap_con = initialize_ldap_con(self.account, self.account_password)
        changes = {:enabled => Set.new, :disabled => Set.new}

        if settings[:account_flags].blank?
          changes[:enabled] = find_all_users(ldap_con, [:login])
        else
          find_all_users(ldap_con, [:login, :account_flags]) do |entry|
            if entry[:login]== '00001276'
              puts "debug user 00001276 #{entry.inspect}"
              puts "00001276 account_disabled #{account_disabled?(entry[:account_flags])}"
            end
            if account_disabled?(entry[:account_flags])
              changes[:disabled] << entry[:login] if entry[:login]
            else
              changes[:enabled] << entry[:login] if entry[:login]
            end
          end
        end

        users_on_local  = self.users.active.map {|u| u.login.downcase }
        puts "-- Found user 00001276 on users_on_local is #{!users_on_local.select {|s| s.include? '00001276'}.nil?}"
        users_on_ldap   = changes.values.sum.map(&:downcase)
        puts "-- Found user 00001276 on users_on_ldap is #{!users_on_ldap.select {|s| s.include? '00001276'}.nil?}"
        deleted_users   = users_on_local - users_on_ldap
        puts "-- deleted_users is #{deleted_users.inspect}"
        changes[:disabled]  += deleted_users
        puts "-- changes[:disabled] for 00001276 is #{changes[:disabled].select {|s| s.include? '00001276'}.inspect}"

        msg = "-- Found #{changes[:enabled].size} users active"
        msg << ", #{changes[:disabled].size - deleted_users.size} locked"
        msg << " and #{deleted_users.size} deleted on ldap"
        puts msg

        @ldap_users = changes
        nil # This will cause an undefined method '[]' for nil:NilClass Error
      end

Output:
/.rvm/rubies/ruby-1.9.3-p327/bin/ruby -e at_exit{sleep(1)};$stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) /home/sidfunktion/.rvm/gems/ruby-1.9.3-p327/gems/ruby-debug-ide-0.4.17.beta16/bin/rdebug-ide --port 45121 --dispatcher-port 50448 -- /.rvm/gems/ruby-1.9.3-p327/bin/rake redmine:plugins:redmine_ldap_sync:sync_users ACTIVATE_USERS=1 --trace
Fast Debugger (ruby-debug-ide 0.4.17.beta16, ruby-debug-base19x 0.11.30.pre11) listens on 127.0.0.1:45121
Fast Debugger (ruby-debug-ide 0.4.17.beta16, ruby-debug-base19x 0.11.30.pre11) listens on 127.0.0.1:56501

** Invoke redmine:plugins:redmine_ldap_sync:sync_users (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute redmine:plugins:redmine_ldap_sync:sync_users
Synchronizing AuthSource dsv-ldap...
debug user 00001276 {:user_memberid=>"cn=00001276,ou=S,o=DSV", :account_flags=>"FALSE", :login=>"00001276", :groupname=>"00001276"}
00001276 account_disabled TRUE
-- Found user 00001276 on users_on_local is true
-- Found user 00001276 on users_on_ldap is true
-- deleted_users is []
-- changes[:disabled] for 00001276 is ["00001276"]
-- Found 321 users active, 1421 locked and 0 deleted on ldap

Result:

def account_disabled?(flags)
...............
return @account_disabled_test.call(flags) if @account_disabled_test > RETURNS TRUE !!!!!!!

from redmine_ldap_sync.

Nice work.

Have you changed the case of the 'true' of the line: '00001276 account_disabled TRUE'?
On a normal output that value is downcased.

I'll try to dig a bit deeper on that and see what might be failing.
The evaluation of the expression is very straight forward and that result is quite strange?

As a side comment: The result of select is never nil.
Saying so, the expression: !users_on_local.select {|s| s.include? '00001276'}.nil? will always return true.
What you probably want is: users_on_local.include?('00001276')

from redmine_ldap_sync.

Make sure you have:
flags == 'TRUE'

Notice the two equals.

Sorry, it seems it was me who have driven you into the mistake of using a simple equal on the expression.

from redmine_ldap_sync.

~ß?#.'!*` >-<

Ok, no it works. All user get activated and no mkore locked again.
I already had a feeling about that is was such silly and small cfg thing and not the code.
Can u please add a note an wiki to remember. This config would work for most novell ldap servers.

Tx alot

!!! CLOSED !!!

from redmine_ldap_sync.

It was a mixture of problems. Encoding and configuration.

Thank you for your patience.

from redmine_ldap_sync.