Updating CPAN Modules

It looks to me like your version of manager on the system doesn't match the version of code on the firewall. We are currently in the process of redoing our documentation, as some fairly major changes have happened. Please use this documentation for now http://www.honeyclient.org/trac/wiki/UserGuide-1.0

But there is one modification. Where it says to do svn co svn://svn.honeyclient.org/honeyclient/tags/rel/1.0.1 honeyclient do instead svn co svn://svn.honeyclient.org/honeyclient/trunk honeyclient

as we've not yet frozen the code into a 1.0.1 release.

Xeno

I checked out the trunk branch, is there any documentation for compiling it?

Replying to xkovah:

It looks to me like your version of manager on the system doesn't match the version of code on the firewall. We are currently in the process of redoing our documentation, as some fairly major changes have happened. Please use this documentation for now http://www.honeyclient.org/trac/wiki/UserGuide-1.0 But there is one modification. Where it says to do svn co svn://svn.honeyclient.org/honeyclient/tags/rel/1.0.1 honeyclient do instead svn co svn://svn.honeyclient.org/honeyclient/trunk honeyclient as we've not yet frozen the code into a 1.0.1 release. Xeno

Nothing should actually need to be compiled in the honeyclient code proper. We don't use XS or anything like that currently, so our code is strictly perl-only, so the previous instructions on making the packages was mostly a means by which to have it do the automatic checks for prerequisites. Therefore, if you have the trunk code checked out, and all the prerequisites installed (and I feel sorry for you manually rather than the new more streamlined CPAN bundles) then you should be able to just run the code. I still recommend just running through those full instructions quickly just to make sure everything is in order though (as there have been additional prerequisites added since the last .tar.gzed version of the code).

Xeno

Oh, but also, note that in the new instructions, we don't actually install the code to anywhere else on the filesystem, so when you run it, rather than issue "perl /usr/bin/StartManager.pl …" like you did above, you need to be in the honeyclient dir which was checked out, and issue the relative path command "perl bin/StartManager.pl …" That's what it shows in that UserGuide-1.0 documentation, but I just thought I would call your attention to it incase that was causing any problems.

Xeno

Hi

I have the same problem when i try to start the Honeyclient Manager.

I did what you say, I downloaded the subversion honeyclient source trunk and I tried to install it as you said, i used http://www.honeyclient.org/trac/wiki/UserGuide-1.0 . I reconfigure CPAN

root@sacarino:~# cat /etc/perl/CPAN/Config.pm …

'cpan_home' ⇒ q/usr/src/HoneyClient-1.0.1/cpan,

…

'urllist' ⇒ [q[=], qfile:///usr/src/HoneyClient-1.0.1/cpan],

… END

And I tried to install it through CPAN but it failed.

cpan[1]> install Bundle::HoneyClient::Util CPAN: Storable loaded ok (v2.15) Going to read /usr/src/HoneyClient-1.0.1/cpan/Metadata

Database was generated on Tue, 26 Feb 2008 22:30:53 GMT

CPAN: YAML loaded ok (v0.66) Going to read root.cpan/build/ ………………………………………………………………….DONE Found 39 old builds, restored the state of 39 Warning: Cannot install Bundle::HoneyClient::Util, don't know what it is. Try the command

i /Bundle::HoneyClient::Util/

to find objects with matching identifiers. CPAN: Time::HiRes loaded ok (v1.86)

cpan[2]>

I think that i installed all dependences and Scalar::Util and all i needed, i think, but i could not install HoneyClient correctly.

Could you give me any idea of what is going wrong?

It looks like your 'urllist' setting has an extra equals (=) in it. I'm thinking that this line:

'urllist' ⇒ [q[=], q[file:///usr/src/HoneyClient-1.0.1/cpan],]

should be, in fact:

'urllist' ⇒ [q[file:///usr/src/HoneyClient-1.0.1/cpan],]

Xeno, do you think this following line is getting improperly parsed?

cpan> o conf urllist = file:///home/user/honeyclient/cpan

Hi

I did this but not resolve the problem. I have the same errors.

A greeting

same problem here…

Oops, I thought I replied yesterday, but apparently it was only in my head

First, about the extra =, CPAN unfortunately uses different syntax for manually setting o conf values depending on whether Bundle::CPAN is already installed or not (specifically I think because Bundle::CPAN installs Module::Build). If you did the instructions as posted and got the extra = that is probably just because of the fact that there was a preexisting install which I assume already had Bundle::CPAN installed. So getting rid of the extra = is desirable, but shouldn't cause a failure, since cpan is smart enough that it can't find a file at a path/URL given in the urllist (the = in this case) it will just fail over to trying the next path. So Darien, I don't think the command as given in the instructions is incorrect, as it's meant to be done with a fresh install, and ultimately it won't be the culprit of actual failures even if the extra = gets in there.

To me, the more likely culprit is the presense of /usr/src/HoneyClient-1.0.1/cpan/Metadata file. Do you know if this file was generated fresh after you re-checked out trunk, or whether it is left over from the previous install? Because the Metadata file basically reads in honeyclient/cpan/sources/modules/02packages.details.txt.gz and creates a cache of it, and if the cache reflects a different 02packages file than the one in our trunk, it will not contain the correct path for the Bundle::HoneyClient::* packages.

So ultimately, the best way to check whether it should be installing correctly is to go "gunzip 02packages.details.txt.gz" (it will decompress in place) and look through it to see if "Bundle::HoneyClient::Util" and the other 2 are in there. If they're not there you need to make sure your "svn info" points to our trunk, and then do an svn up. If they are there, then it's probably the Metadata file's fault, so gzip the .txt back up (again it will do it in place) and to be safe just do an "rm -rf cpan/build" "rm -rf cpan/Bundle" "rm cpan/Metadata" and then "svn up" which will restore the directories. Try the cpan command "install Bundle::HoneyClient::Util" again, and you should see cpan generating the fresh Metadata file, before subsequently executing the install command and working or failing.

IF that then fails, I recommend really going back to basics with the cpan configuration by wiping it out and configuring it from the guide again. To find where the actual Config.pm file is stored you can do an "o conf commit" from the cpan prompt and it will say something like this: "commit: wrote '/etc/perl/CPAN/Config.pm'" and that's then the file to wipe. Unfortunately, when you go back into cpan for the first time, since you have Module::Build installed it may not exactly line up with the instructions 1:1 since Module::Build wants you to set extra options. However, as before, the 2 main options to set are the cpan_home and the urllist.

Let us know if that works.

Xeno

I tried a completely new install from scratch, and it seems to work. I will finish it next week, i'll give you complete feedback.

We just finished the UserGuide updates and have release version 1.0.2 of the honeyclient and drone packages. Please see the main UserGuide for information about how to install using this code base.

If you have any problems or questions, please let us know.

Thanks!

On Monday with trunk it worked, today with 1.0.2 it does not work again

install Bundle::HoneyClient::Util
CPAN: Storable loaded ok (v2.15)
Going to read /home/honeyusr/honeyclient/cpan/Metadata
  Database was generated on Wed, 05 Mar 2008 11:31:04 GMT
CPAN: YAML loaded ok (v0.66)
Going to read /home/honeyusr/honeyclient/cpan/build/
DONE
Found no old builds, restored the state of none
Warning: Cannot install Bundle::HoneyClient::Util, don't know what it is.
Try the command

    i /Bundle::HoneyClient::Util/

to find objects with matching identifiers.
CPAN: Time::HiRes loaded ok (v1.86)

Config.pm

  'urllist' => [q[=], q[file:///home/honeyusr/honeyclient/cpan]],

tried it without the addition q[=], same result.

Replying to kindlund:

We just finished the UserGuide updates and have release version 1.0.2 of the honeyclient and drone packages. Please see the main UserGuide for information about how to install using this code base. If you have any problems or questions, please let us know. Thanks!

the post above is mine…

Apparently my email reply didn't get attached to trac. Basically an incorrect version of the code was accidentally copied over to that tag. You can either still use trunk, or use the copy from the downloads page. Darien will be resolving this soon.

Xeno

Okay, this issue has been fixed; sorry about that. You should get the right code, by doing:

svn co svn://svn.honeyclient.org/honeyclient/tags/rel/1.0.1

Or, by downloading the honeyclient-1.0.2.tar.gz package on the downloads page.

Please let us know if you run into any other issues.

Thanks,

— Darien

Hi,

now i've some problems with the drone support:

root@honey3:~/drone# RAILS_ENV=production rake db:migrate
(in /home/honeyusr/drone)
FasterCSV is not installed. CSV functionality will not be included.
rake aborted!
#42000Unknown database 'drone_production'

(See full trace by running task with --trace)
root@honey3:~/drone# rake db:migrate

I've no idea about ruby and mysql, so i've no clue…

I recognized an error in the new user guide:

3. Next, add the bin/bootstrap_agent.sh script to your ~/.bashrc file, in order to have the Agent execute every time a Cygwin shell is opened. You can accomplish this by typing the following: 
$ echo "./run.sh" >> ~/.bashrc

should be

$ echo "~/honeyclient/bin/bootstrap_agent.sh" >> ~/.bashrc

Hi,

For the first issue:

Unknown database 'drone_production'

That means your MySQL server does not contain a database named 'drone_production'. This database should have been created, when you ran the following:

$ cd ~/drone
$ RAILS_ENV=production rake db:create

When you perform this command, does rake return any sort of error message? After performing this command, can you connect to the mysql server and verify that a database named 'drone_production' was created? You can do this, by simply typing the following commands from the command line of the drone system:

$ mysql -u <root_user> -p
Enter password: <root_pass>
...
mysql> show databases;
+-------------------------+
| Database                |
+-------------------------+
| drone_development       |
| drone_production        |
| drone_test              |
| mysql                   |
+-------------------------+
...
mysql> exit

If you see the 'drone_production' database listed, then you should be able to run the rake db:migrate command without any problems. If you're still running into problems, please let us know.

Lastly, thanks for pointing out the error in the UserGuide… I'll update it right now.

Regards,

— Darien

…. seems to be yet another stupid c&p error…

i thought i ran the rake db:create cmd and wondered about the missing database, but when i'm no looking at my post….

thank you one more time, i'll try it…

Hi,

got drone finally installed without any problems

So i tested to run HoneyClient with the two example urls from honeyclient/bin/insert_queue_urls.pl.

Now the process seems to be stuck, i cannot detect any activity:

root@honey3:~/honeyclient# perl -Ilib bin/StartManager.pl
Starting new session...
2008-03-07 11:23:57  INFO [HoneyClient::Manager::VM::init] (lib/HoneyClient/Manager/VM.pm:757) - Initializing VM daemon at PID: 11978
2008-03-07 11:23:57  INFO [HoneyClient::Manager::VM::Clone::new] (lib/HoneyClient/Manager/VM/Clone.pm:885) - Setting VM (/home/honeyusr/vm/master/master.vmx) as master.
2008-03-07 11:24:10  INFO [HoneyClient::Manager::VM::Clone::_init] (lib/HoneyClient/Manager/VM/Clone.pm:580) - Quick cloning master VM (/home/honeyusr/vm/master/master.vmx).

Okay, from the output you've provided, it looks like the Manager is still in the process of cloning the master VM…

1. Is there any additional output that the system returns, after that last message? Or does it just hang indefinitely?

The problem is that the Manager won't fetch URLs from the database until after this clone is successfully created. If you're having problems creating a clone, then there might be something else going on.

2. What are the specifications of the host system? CPU/RAM/HD size would be helpful.

3. While you execute this command, can you login to the host system using the VMware Server Console as root and let me know if you see any activity? At that step, you should be able to see the cloned VM created and booting up — perhaps the Agent process did not start properly within the cloned VM, when you booted it up?

Any additional information that you can provide would be helpful, in order to troubleshoot further.

— Darien

I have seen something similar to this before. If you log into the vmware console and DO see that the agent seems to have been started successfully, I recommend going back and checking that between reboots that your /var/log/messages file still has the same permissions as specified in the guide, and that your syslog is correctly logging all the vmware information to /var/log/messages per the modified syslog configuration file. In my previous experience when I tried to install on ubuntu, the vmware messages related to DHCP lease information, which honeyclient depends on to determine the IP address of the Agent, were getting logged to some other file than /var/log/messages, and hence the Manager never got to the subsequent step where it gets the IP and tries to connect to the Agent.

Hi,

ok, but maybe i should start a new ticket?

I watched the dir where the cloned VMs are stored, and the new clone of the master vm is created, but the performance is just about 1MiB/Minute. That's why i thought it was stucked after 1,5 days

Infos about the system: * There was no more output after this message (not weird with that new informations ) * P4, 2,6Ghz, 1GB RAM, 60GB HD, 40GB available * I still run Ubuntu 7.10 Server * When i log in with vmware console remote, i don't see the cloned vm…

I think there's another problem now about the bad write performance…

Good idea. The initial issue has been resolved, so I'm closing this ticket. For further discussion, see ticket #148. Please reference this ticket from now on.