zabak / google-sites-mirror Goto Github PK

What steps will reproduce the problem?
# /usr/bin/python2.6 /usr/bin/gsmirror -s production-support -d bla.com -p 
/backup/google-sites/20130906/ -t /backup/google-sites/default_templates/ -n 
production-support --email [email protected] --password "pass" -l

What is the expected output? What do you see instead?

Expect download with sucess, but I got:

downloading page: blacklistprocessor
downloading ocontent of: blacklistprocessor
downloading attachment information: 5-ConsoleOutput.jpg
downloading attachment information: 3-BuildNow.jpg
downloading attachment information: 1-jenkins_views
downloading attachment information: 4-build45.jpg
downloading attachment information: 2-BlacklistProcessorJob.jpg
downloading attachment information: 1-JenkinsViews.jpg
downloading page: siteupdater
downloading ocontent of: siteupdater
downloading page: tasks
downloading ocontent of: tasks
downloading list item at: tasks
saving page: home
Traceback (most recent call last):
  File "/usr/bin/gsmirror", line 9, in <module>
    load_entry_point('gsmirror==1.0.2', 'console_scripts', 'gsmirror')()
  File "/usr/lib/python2.6/site-packages/gsmirror-1.0.2-py2.6.egg/gsmirroring/gsmirror.py", line 137, in main
    siteController.save_site_to_disk(site=site, path=path, directory=name)
  File "/usr/lib/python2.6/site-packages/gsmirror-1.0.2-py2.6.egg/gsmirroring/site_controller.py", line 273, in save_site_to_disk
    self.save_page_to_disk(page=child, path=path_to_site, site=site)
  File "/usr/lib/python2.6/site-packages/gsmirror-1.0.2-py2.6.egg/gsmirroring/site_controller.py", line 327, in save_page_to_disk
    self.save_as_file(content=str(template_out), path=page_path + PAGE_NAME)
  File "/usr/lib64/python2.6/site-packages/Cheetah/Template.py", line 1005, in __str__
    rc = getattr(self, mainMethName)()
  File "DynamicallyCompiledCheetahTemplate.py", line 89, in respond
NameError: global name 'node' is not defined



What version of the product are you using? On what operating system?
Red Hat Enterprise Linux Server release 6.3 (Santiago)
gsmirror-1.0.2

Please provide any additional information below.

What steps will reproduce the problem?
1. Trying to mirror my site


What is the expected output? What do you see instead?

I used to get the expected "downloading site" prompt. But now I get the 
following error

Traceback (most recent call last):
  File "/usr/bin/gsmirror", line 9, in <module>
    load_entry_point('gsmirror==1.0.2', 'console_scripts', 'gsmirror')()
  File "/usr/lib/python2.7/site-packages/gsmirror-1.0.2-py2.7.egg/gsmirroring/gsmirror.py", line 136, in main
    site = siteController.get_site()
  File "/usr/lib/python2.7/site-packages/gsmirror-1.0.2-py2.7.egg/gsmirroring/site_controller.py", line 380, in get_site
    self.get_page(entry, level=1, parent=site)
  File "/usr/lib/python2.7/site-packages/gsmirror-1.0.2-py2.7.egg/gsmirroring/site_controller.py", line 417, in get_page
    author_name=entry.author[0].name.text,
AttributeError: 'NoneType' object has no attribute 'text'


What version of the product are you using? On what operating system?

gsmirror-1.0.2.zip

Please provide any additional information below.

What steps will reproduce the problem?
1. Exporting a site with gadgets such as Table of Contents 


What is the expected output? What do you see instead?

I would like to see a rendered table of contents, instead I just see a grey 
box. Is it possible for data from gadgets to be exported or rendered in a 
different way?


What version of the product are you using? On what operating system?

1.0.2 on OS X 10.6.5

What steps will reproduce the problem?
1. Run gsmirror provided with appropriate extensions
2. Downloading of sites structure started
3. Error in building sites output

What is the expected output? What do you see instead?
I expected that my google site would be mirrored.

What version of the product are you using? On what operating system?
gsmirror-1.0.2
Ubuntu Linux 10.04 LTS

Please provide any additional information below.

  File "/usr/local/bin/gsmirror", line 8, in <module>
    load_entry_point('gsmirror==1.0.2', 'console_scripts', 'gsmirror')()
  File "build/bdist.linux-x86_64/egg/gsmirroring/gsmirror.py", line 136, in main
  File "build/bdist.linux-x86_64/egg/gsmirroring/site_controller.py", line 380, in get_site
  File "build/bdist.linux-x86_64/egg/gsmirroring/site_controller.py", line 436, in get_page
  File "build/bdist.linux-x86_64/egg/gsmirroring/site_controller.py", line 398, in get_site_content
  File "build/bdist.linux-x86_64/egg/gsmirroring/site_controller.py", line 436, in get_page
  File "build/bdist.linux-x86_64/egg/gsmirroring/site_controller.py", line 400, in get_site_content
  File "build/bdist.linux-x86_64/egg/gsmirroring/site_controller.py", line 474, in get_attachment
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2013' in position 
55: ordinal not in range(128)

What steps will reproduce the problem?
1. Try to mirror one website with password
2. /usr/bin/python /usr/bin/gsmirror -s alert-procedures -d bla.com -p 
/backup/google-sites/20130903/ -t /backup/google-sites/default_templates/ -n 
alert-procedures --email [email protected] --password bla -l 

What is the expected output? What do you see instead?
downloading page: infra_p0_check_time_zone
downloading ocontent of: infra_p0_check_time_zone
downloading page: ops_p1_sbs_chargeback_configuration_and_renowable
downloading ocontent of: ops_p1_sbs_chargeback_configuration_and_renowable
downloading page: diveo_diveo13_l
downloading ocontent of: diveo_diveo13_l
downloading page: ops_p1_webservice_purebros
downloading ocontent of: ops_p1_webservice_purebros
downloading page: ops_p0_subscription_engine_jms_queue
downloading ocontent of: ops_p0_subscription_engine_jms_queue
downloading page: ops_p1_webservice_vivo
downloading ocontent of: ops_p1_webservice_vivo
downloading page: ops_p1_webservice_claro
downloading ocontent of: ops_p1_webservice_claro
downloading page: ops_p1_webservice_acotel
downloading ocontent of: ops_p1_webservice_acotel
downloading page: ops_p1_sbs_jmx_num_publications
downloading ocontent of: ops_p1_sbs_jmx_num_publications
downloading page: ops_p0_jmx_owl
downloading ocontent of: ops_p0_jmx_owl
downloading page: ops_p1_http_29003_url
downloading ocontent of: ops_p1_http_29003_url
downloading page: ops_p1_mo_cancel_fallback
downloading ocontent of: ops_p1_mo_cancel_fallback
downloading page: ops_p0_billingconsumer_jms_queue
downloading ocontent of: ops_p0_billingconsumer_jms_queue
downloading page: ops_p0_api_sbs_engine_process
downloading ocontent of: ops_p0_api_sbs_engine_process
downloading page: ops_p0_api_sbs_consumer_process
downloading ocontent of: ops_p0_api_sbs_consumer_process
downloading page: infra_p0_check_time_zone
downloading ocontent of: infra_p0_check_time_zone
downloading page: ops_p0_subscription_process
downloading ocontent of: ops_p0_subscription_process
downloading page: proceduretemplate
downloading ocontent of: proceduretemplate
saving page: home
/usr/lib/python2.6/site-packages/Cheetah-2.4.4-py2.6.egg/Cheetah/Compiler.py:150
9: UserWarning: 
You don't have the C version of NameMapper installed! I'm disabling Cheetah's 
useStackFrames option as it is painfully slow with the Python version of 
NameMapper. You should get a copy of Cheetah with the compiled C version of 
NameMapper.
  "\nYou don't have the C version of NameMapper installed! "

Strace:
close(3)                                = 0
gettimeofday({1378224566, 864321}, NULL) = 0
socket(PF_FILE, SOCK_DGRAM|SOCK_CLOEXEC, 0) = 3
connect(3, {sa_family=AF_FILE, path="/dev/log"}, 110) = 0
sendto(3, "<14>Sep  3 16:09:26 abrt: detect"..., 84, MSG_NOSIGNAL, NULL, 0) = 84
stat("/usr/bin/gsmirror", {st_mode=S_IFREG|0755, st_size=306, ...}) = 0
open("/usr/bin/gsmirror", O_RDONLY)     = 4
fstat(4, {st_mode=S_IFREG|0755, st_size=306, ...}) = 0
fstat(4, {st_mode=S_IFREG|0755, st_size=306, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 
0x7f8862918000
read(4, "#!/usr/bin/python\n# EASY-INSTALL"..., 8192) = 306
read(4, "", 4096)                       = 0
close(4)                                = 0
munmap(0x7f8862918000, 4096)            = 0
stat("/usr/lib/python2.6/site-packages/gsmirror-1.0.2-py2.6.egg/gsmirroring/gsmi
rror.py", {st_mode=S_IFREG|0755, st_size=4979, ...}) = 0
open("/usr/lib/python2.6/site-packages/gsmirror-1.0.2-py2.6.egg/gsmirroring/gsmi
rror.py", O_RDONLY) = 4
fstat(4, {st_mode=S_IFREG|0755, st_size=4979, ...}) = 0
fstat(4, {st_mode=S_IFREG|0755, st_size=4979, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 
0x7f8862918000
read(4, "#! /usr/bin/python\n\n\n__author__="..., 8192) = 4979
read(4, "", 4096)                       = 0
close(4)                                = 0
munmap(0x7f8862918000, 4096)            = 0
stat("/usr/lib/python2.6/site-packages/gsmirror-1.0.2-py2.6.egg/gsmirroring/site
_controller.py", {st_mode=S_IFREG|0755, st_size=19912, ...}) = 0
open("/usr/lib/python2.6/site-packages/gsmirror-1.0.2-py2.6.egg/gsmirroring/site
_controller.py", O_RDONLY) = 4
fstat(4, {st_mode=S_IFREG|0755, st_size=19912, ...}) = 0
fstat(4, {st_mode=S_IFREG|0755, st_size=19912, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 
0x7f8862918000
read(4, "__author__=\"hanis (Jan Rychtar)\""..., 8192) = 8192
read(4, "+9:-1]\n                    new_l"..., 4096) = 4096
read(4, " in page.childs:\n            sel"..., 4096) = 4096
read(4, "  content_str = self.request_pag"..., 4096) = 3528
read(4, "", 4096)                       = 0
close(4)                                = 0
munmap(0x7f8862918000, 4096)            = 0
stat("/usr/lib/python2.6/site-packages/gsmirror-1.0.2-py2.6.egg/gsmirroring/site
_controller.py", {st_mode=S_IFREG|0755, st_size=19912, ...}) = 0
stat("/usr/lib/python2.6/site-packages/Cheetah-2.4.4-py2.6.egg/Cheetah/Template.
py", {st_mode=S_IFREG|0755, st_size=85089, ...}) = 0
open("/usr/lib/python2.6/site-packages/Cheetah-2.4.4-py2.6.egg/Cheetah/Template.
py", O_RDONLY) = 4
fstat(4, {st_mode=S_IFREG|0755, st_size=85089, ...}) = 0
fstat(4, {st_mode=S_IFREG|0755, st_size=85089, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 
0x7f8862918000
read(4, "'''\nProvides the core API for Ch"..., 8192) = 8192
read(4, "use pattern 1 instead):\n        "..., 4096) = 4096
read(4, " _getCompilerSettings(klass, sou"..., 4096) = 4096
read(4, "eady in\n              sys.module"..., 4096) = 4096
read(4, " by zero\n            \n          "..., 4096) = 4096
read(4, "ngs.\n                   - search"..., 4096) = 4096
read(4, "   if not isinstance(cacheModule"..., 4096) = 4096
read(4, "== 'ascii':\n                gene"..., 4096) = 4096
read(4, "this is T2.meth1')\n        \"\"\"\n "..., 4096) = 4096
read(4, "   if not hasattr(settings, 'com"..., 4096) = 4096
read(4, "   def __str__(self):\n          "..., 4096) = 4096
read(4, " Users' Guide\n               for"..., 4096) = 4096
read(4, "g %\n                            "..., 4096) = 4096
read(4, "acheStore = self._CHEETAH_cacheS"..., 4096) = 4096
read(4, "   Other notes:\n        - There "..., 4096) = 4096
read(4, "e(errorCatcher, basestring):\n   "..., 4096) = 4096
read(4, "ation so compilerSettings\n      "..., 4096) = 4096
read(4, "nt.  So the debugging\n        ou"..., 4096) = 4096
read(4, "code.  Example:\n\n            #si"..., 4096) = 4096
read(4, " a subclass for Cheetah.\n       "..., 4096) = 3169
read(4, "", 4096)                       = 0
close(4)                                = 0
munmap(0x7f8862918000, 4096)            = 0
stat("DynamicallyCompiledCheetahTemplate.py", 0x7fffa37c5200) = -1 ENOENT (No 
such file or directory)
stat("/usr/bin/DynamicallyCompiledCheetahTemplate.py", 0x7fffa37c5200) = -1 
ENOENT (No such file or directory)
stat("/usr/lib/python2.6/site-packages/gsmirror-1.0.2-py2.6.egg/DynamicallyCompi
ledCheetahTemplate.py", 0x7fffa37c5200) = -1 ENOENT (No such file or directory)
stat("/usr/lib/python2.6/site-packages/gdata-2.0.17-py2.6.egg/DynamicallyCompile
dCheetahTemplate.py", 0x7fffa37c5200) = -1 ENOENT (No such file or directory)
stat("/usr/lib/python2.6/site-packages/BeautifulSoup-3.2.1-py2.6.egg/Dynamically
CompiledCheetahTemplate.py", 0x7fffa37c5200) = -1 ENOENT (No such file or 
directory)
stat("/usr/lib/python2.6/site-packages/Cheetah-2.4.4-py2.6.egg/DynamicallyCompil
edCheetahTemplate.py", 0x7fffa37c5200) = -1 ENOENT (No such file or directory)
stat("/usr/lib/python2.6/site-packages/Markdown-2.3.1-py2.6.egg/DynamicallyCompi
ledCheetahTemplate.py", 0x7fffa37c5200) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python26.zip/DynamicallyCompiledCheetahTemplate.py", 
0x7fffa37c5200) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python2.6/DynamicallyCompiledCheetahTemplate.py", 
0x7fffa37c5200) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python2.6/plat-linux2/DynamicallyCompiledCheetahTemplate.py", 
0x7fffa37c5200) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python2.6/lib-tk/DynamicallyCompiledCheetahTemplate.py", 
0x7fffa37c5200) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python2.6/lib-old/DynamicallyCompiledCheetahTemplate.py", 
0x7fffa37c5200) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python2.6/lib-dynload/DynamicallyCompiledCheetahTemplate.py", 
0x7fffa37c5200) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python2.6/site-packages/DynamicallyCompiledCheetahTemplate.py",
 0x7fffa37c5200) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python2.6/site-packages/gtk-2.0/DynamicallyCompiledCheetahTempl
ate.py", 0x7fffa37c5200) = -1 ENOENT (No such file or directory)
stat("/usr/lib/python2.6/site-packages/DynamicallyCompiledCheetahTemplate.py", 
0x7fffa37c5200) = -1 ENOENT (No such file or directory)
stat("/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info/Dynamica
llyCompiledCheetahTemplate.py", 0x7fffa37c5200) = -1 ENOTDIR (Not a directory)
stat("/usr/bin/gsmirror", {st_mode=S_IFREG|0755, st_size=306, ...}) = 0
stat("/usr/lib/python2.6/site-packages/gsmirror-1.0.2-py2.6.egg/gsmirroring/gsmi
rror.py", {st_mode=S_IFREG|0755, st_size=4979, ...}) = 0
stat("/usr/lib/python2.6/site-packages/gsmirror-1.0.2-py2.6.egg/gsmirroring/site
_controller.py", {st_mode=S_IFREG|0755, st_size=19912, ...}) = 0
stat("/usr/lib/python2.6/site-packages/gsmirror-1.0.2-py2.6.egg/gsmirroring/site
_controller.py", {st_mode=S_IFREG|0755, st_size=19912, ...}) = 0
stat("/usr/lib/python2.6/site-packages/Cheetah-2.4.4-py2.6.egg/Cheetah/Template.
py", {st_mode=S_IFREG|0755, st_size=85089, ...}) = 0
stat("DynamicallyCompiledCheetahTemplate.py", 0x7fffa37c5640) = -1 ENOENT (No 
such file or directory)
stat("/usr/bin/DynamicallyCompiledCheetahTemplate.py", 0x7fffa37c5640) = -1 
ENOENT (No such file or directory)
stat("/usr/lib/python2.6/site-packages/gsmirror-1.0.2-py2.6.egg/DynamicallyCompi
ledCheetahTemplate.py", 0x7fffa37c5640) = -1 ENOENT (No such file or directory)
stat("/usr/lib/python2.6/site-packages/gdata-2.0.17-py2.6.egg/DynamicallyCompile
dCheetahTemplate.py", 0x7fffa37c5640) = -1 ENOENT (No such file or directory)
stat("/usr/lib/python2.6/site-packages/BeautifulSoup-3.2.1-py2.6.egg/Dynamically
CompiledCheetahTemplate.py", 0x7fffa37c5640) = -1 ENOENT (No such file or 
directory)
stat("/usr/lib/python2.6/site-packages/Cheetah-2.4.4-py2.6.egg/DynamicallyCompil
edCheetahTemplate.py", 0x7fffa37c5640) = -1 ENOENT (No such file or directory)
stat("/usr/lib/python2.6/site-packages/Markdown-2.3.1-py2.6.egg/DynamicallyCompi
ledCheetahTemplate.py", 0x7fffa37c5640) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python26.zip/DynamicallyCompiledCheetahTemplate.py", 
0x7fffa37c5640) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python2.6/DynamicallyCompiledCheetahTemplate.py", 
0x7fffa37c5640) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python2.6/plat-linux2/DynamicallyCompiledCheetahTemplate.py", 
0x7fffa37c5640) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python2.6/lib-tk/DynamicallyCompiledCheetahTemplate.py", 
0x7fffa37c5640) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python2.6/lib-old/DynamicallyCompiledCheetahTemplate.py", 
0x7fffa37c5640) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python2.6/lib-dynload/DynamicallyCompiledCheetahTemplate.py", 
0x7fffa37c5640) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python2.6/site-packages/DynamicallyCompiledCheetahTemplate.py",
 0x7fffa37c5640) = -1 ENOENT (No such file or directory)
stat("/usr/lib64/python2.6/site-packages/gtk-2.0/DynamicallyCompiledCheetahTempl
ate.py", 0x7fffa37c5640) = -1 ENOENT (No such file or directory)
stat("/usr/lib/python2.6/site-packages/DynamicallyCompiledCheetahTemplate.py", 
0x7fffa37c5640) = -1 ENOENT (No such file or directory)
stat("/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info/Dynamica
llyCompiledCheetahTemplate.py", 0x7fffa37c5640) = -1 ENOTDIR (Not a directory)
socket(PF_FILE, SOCK_STREAM, 0)         = 4
connect(4, {sa_family=AF_FILE, path="/var/run/abrt/abrt.socket"}, 27



What version of the product are you using? On what operating system?
gsmirror 1.0.2
Red Hat Enterprise Linux Server release 6.3 (Santiago)


Please provide any additional information below.

the process become freeze (I've waited for one week, and then killed)

How can one backup a private site?

What steps will reproduce the problem?
1. run gsmirror

What is the expected output? What do you see instead?
expected: download ok
got: gdata.client.RequestError: Server responded with: 500, Internal Error


What version of the product are you using? On what operating system?
gsmirror-1.0.2
centos 5
centos 6
ubuntu 12.04

Please provide any additional information below.

$ gsmirror -s systems -d site.com -p /backup/20140430/ -t 
/backup/google-sites/default_templates/ -n systems --email [email protected] 
--password 'bleblable' -l
downloading site
Traceback (most recent call last):
  File "/usr/local/bin/gsmirror", line 9, in <module>
    load_entry_point('gsmirror==1.0.2', 'console_scripts', 'gsmirror')()
  File "/usr/local/lib/python2.7/dist-packages/gsmirror-1.0.2-py2.7.egg/gsmirroring/gsmirror.py", line 136, in main
    site = siteController.get_site()
  File "/usr/local/lib/python2.7/dist-packages/gsmirror-1.0.2-py2.7.egg/gsmirroring/site_controller.py", line 375, in get_site
    feed = self.client.GetContentFeed(uri=uri)
  File "/usr/local/lib/python2.7/dist-packages/gdata-2.0.17-py2.7.egg/gdata/sites/client.py", line 155, in get_content_feed
    auth_token=auth_token, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/gdata-2.0.17-py2.7.egg/gdata/client.py", line 640, in get_feed
    **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/gdata-2.0.17-py2.7.egg/gdata/client.py", line 319, in request
    RequestError)
gdata.client.RequestError: Server responded with: 500, Internal Error

What steps will reproduce the problem?
1. Download a page with an attachment that includes an accented character
in the name (e.g.: "á.txt")

What is the expected output? What do you see instead?
- Expected output: full site stored in the output folder
- Actual output: 

Traceback (most recent call last):
  File "C:\Python26\Scripts\gsmirror-script.py", line 8, in <module>
    load_entry_point('gsmirror==1.0.2', 'console_scripts', 'gsmirror')()
  File "build\bdist.win32\egg\gsmirroring\gsmirror.py", line 137, in main
  File "build\bdist.win32\egg\gsmirroring\site_controller.py", line 273, in
save
_site_to_disk
  File "build\bdist.win32\egg\gsmirroring\site_controller.py", line 333, in
save
_page_to_disk
  File "build\bdist.win32\egg\gsmirroring\site_controller.py", line 330, in
save
_page_to_disk
  File "build\bdist.win32\egg\gsmirroring\site_controller.py", line 342, in
save
_attachment_to_file
  File "build\bdist.win32\egg\gsmirroring\xml_module.py", line 124, in
add_eleme
nt
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe1' in
position 55:
 ordinal not in range(128)

What version of the product are you using? On what operating system?
- gsmirror-1.0.2.zip
- Windows XP

Please provide any additional information below.
- Try to encode the file name to UTF-8 in xml_module.py:124