How to mass download FTP site using wget ?

How to mass download FTP site using wget ?

I’m using the Cloud Sites from RackSpace, too bad that they do not provide SSH for me, and I’m going to move my site to the server, I’m using this way to download all the files using wget mass download.

Note:
The -r is recursive
The -l99, I put this as much as possible for the sub directory in the path.

wget -r -l99 ftp://username:[email protected]/public_html/

How to exclude files in wget while mass download files from FTP ?


wget -r -l99 ftp://username:[email protected]/public_html/ -X "/ftp.sample.com/public_html/cache/"

Common Error in CentOS: Failed to set locale, defaulting to C

Failed to set locale, defaulting to C

You can see this often in CentOS because you never set the locale for your Linux.
You can try this
[code lang=”bash”]
echo "export LC_ALL=en_US.UTF-8" >> /etc/profile
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8
export LC_COLLATE=C
export LC_CTYPE=en_US.UTF-8
[/code]
Then disconnect and reconnect to your bash.

MongoDB Error: exception in initAndListen std::exception: boost::filesystem::exists: Permission denied: “/sys/devices/system/node/node1”

I found this error while installed
yum install mongo-10gen-2.2.3 mongo-10gen-server-2.2.3
in CentOS 6.4 64bit


Wed Oct 16 16:19:08 [initandlisten] MongoDB starting : pid=22457 port=27017 dbpath=/var/lib/mongo 64-bit host=xxxxxxx.net
Wed Oct 16 16:19:08 [initandlisten] exception in initAndListen std::exception: boost::filesystem::exists: Permission denied: "/sys/devices/system/node/node1", terminating
Wed Oct 16 16:19:08 dbexit:
Wed Oct 16 16:19:08 [initandlisten] shutdown: going to close listening sockets...
Wed Oct 16 16:19:08 [initandlisten] shutdown: going to flush diaglog...
Wed Oct 16 16:19:08 [initandlisten] shutdown: going to close sockets...
Wed Oct 16 16:19:08 [initandlisten] shutdown: waiting for fs preallocator...
Wed Oct 16 16:19:08 [initandlisten] shutdown: lock for final commit...
Wed Oct 16 16:19:08 [initandlisten] shutdown: final commit...
Wed Oct 16 16:19:08 [initandlisten] shutdown: closing all files...
Wed Oct 16 16:19:08 [initandlisten] closeAllFiles() finished
Wed Oct 16 16:19:08 dbexit: really exiting now

To solve this problem temporally I make mongod have READ access to /sys/devices/system/node/node1
chmod 755 /sys/devices/system/node
This solved my problem.


[[email protected] ~]# chmod 755 /sys/devices/system/node
[[email protected] ~]# service mongod restart
Stopping mongod: [FAILED]
Starting mongod: forked process: 23540
all output going to: /var/log/mongo/mongod.log
child process started successfully, parent exiting
[ OK ]
[[email protected] ~]# ls -la /sys/devices/system/node

MongoDB Error: about to fork child process, waiting until server is ready for connections.

I tried to install MongoDB to CentOS 6.4 64Bit and found the error while I launch mongodb like this:
[code lang=”bash”]
numactl –interleave=all /usr/bin/mongod -f /etc/mongod.conf
[/code]
Here are the error code.
[code lang=”bash”]
about to fork child process, waiting until server is ready for connections.
forked process: 9713
Wed Oct 16 02:00:00.640 terminate() called, printing stack (if implemented for platform):
0xdddd81 0x6cfbae 0x35d60203be6 0x35d60203c13 0x35d60203d0e 0x35d601a8ce7 0x35d60201a04 0x35d601ad3bc 0x35d601ae226 0xdfb5df 0xdfbf2b 0xdf8bd0 0x9ed4df 0x6dde80 0x6dfc29 0x35d5f938cdd 0x6cf999
/usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xdddd81]
/usr/bin/mongod(_ZN5mongo11myterminateEv+0x3e) [0x6cfbae]
/usr/lib64/libstdc++.so.6(+0xbcbe6) [0x35d60203be6]
/usr/lib64/libstdc++.so.6(+0xbcc13) [0x35d60203c13]
/usr/lib64/libstdc++.so.6(+0xbcd0e) [0x35d60203d0e]
/usr/lib64/libstdc++.so.6(_ZSt21__throw_runtime_errorPKc+0x67) [0x35d601a8ce7]
/usr/lib64/libstdc++.so.6(+0xbaa04) [0x35d60201a04]
/usr/lib64/libstdc++.so.6(_ZNSt6locale5_ImplC2EPKcm+0x4c) [0x35d601ad3bc]
/usr/lib64/libstdc++.so.6(_ZNSt6localeC2EPKc+0x5f6) [0x35d601ae226]
/usr/bin/mongod(_ZN5boost11filesystem34path21wchar_t_codecvt_facetEv+0x4f) [0xdfb5df]
/usr/bin/mongod(_ZNK5boost11filesystem34path14root_directoryEv+0xbb) [0xdfbf2b]
/usr/bin/mongod(_ZN5boost11filesystem38absoluteERKNS0_4pathES3_+0x40) [0xdf8bd0]
/usr/bin/mongod(_ZN5mongo27initializeServerGlobalStateEb+0x15f) [0x9ed4df]
/usr/bin/mongod() [0x6dde80]
/usr/bin/mongod(main+0x9) [0x6dfc29]
/lib64/libc.so.6(__libc_start_main+0xfd) [0x35d5f938cdd]
/usr/bin/mongod() [0x6cf999]
Wed Oct 16 02:00:00.645 Got signal: 6 (Aborted).

Wed Oct 16 02:00:00.649 Backtrace:
0xdddd81 0x6d0d29 0x35d5f94c960 0x35d5f94c8e5 0x35d5f94e0c5 0x6cfbb3 0x35d60203be6 0x35d60203c13 0x35d60203d0e 0x35d601a8ce7 0x35d60201a04 0x35d601ad3bc 0x35d601ae226 0xdfb5df 0xdfbf2b 0xdf8bd0 0x9ed4df 0x6dde80 0x6dfc29 0x35d5f938cdd
/usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0xdddd81]
/usr/bin/mongod(_ZN5mongo10abruptQuitEi+0x399) [0x6d0d29]
/lib64/libc.so.6(+0x32960) [0x35d5f94c960]
/lib64/libc.so.6(gsignal+0x35) [0x35d5f94c8e5]
/lib64/libc.so.6(abort+0x175) [0x35d5f94e0c5]
/usr/bin/mongod(_ZN5mongo11myterminateEv+0x43) [0x6cfbb3]
/usr/lib64/libstdc++.so.6(+0xbcbe6) [0x35d60203be6]
/usr/lib64/libstdc++.so.6(+0xbcc13) [0x35d60203c13]
/usr/lib64/libstdc++.so.6(+0xbcd0e) [0x35d60203d0e]
/usr/lib64/libstdc++.so.6(_ZSt21__throw_runtime_errorPKc+0x67) [0x35d601a8ce7]
/usr/lib64/libstdc++.so.6(+0xbaa04) [0x35d60201a04]
/usr/lib64/libstdc++.so.6(_ZNSt6locale5_ImplC2EPKcm+0x4c) [0x35d601ad3bc]
/usr/lib64/libstdc++.so.6(_ZNSt6localeC2EPKc+0x5f6) [0x35d601ae226]
/usr/bin/mongod(_ZN5boost11filesystem34path21wchar_t_codecvt_facetEv+0x4f) [0xdfb5df]
/usr/bin/mongod(_ZNK5boost11filesystem34path14root_directoryEv+0xbb) [0xdfbf2b]
/usr/bin/mongod(_ZN5boost11filesystem38absoluteERKNS0_4pathES3_+0x40) [0xdf8bd0]
/usr/bin/mongod(_ZN5mongo27initializeServerGlobalStateEb+0x15f) [0x9ed4df]
/usr/bin/mongod() [0x6dde80]
/usr/bin/mongod(main+0x9) [0x6dfc29]
/lib64/libc.so.6(__libc_start_main+0xfd) [0x35d5f938cdd]

ERROR: child process failed, exited with error number 14
[/code]

Solved

The way i use to solve the problem by this.
[code lang=”bash”]
[[email protected] ~]# export LANGUAGE=en_US.UTF-8
[[email protected] ~]# export LANG=en_US.UTF-8
[[email protected] ~]# export LC_ALL=en_US.UTF-8
[[email protected] ~]#
[[email protected] ~]# numactl –interleave=all /usr/bin/mongod -f /etc/mongod.conf
about to fork child process, waiting until server is ready for connections.
forked process: 9821
all output going to: /var/log/mongo/mongod.log
child process started successfully, parent exiting
[/code]

Optimize Server Hard Disk for Performance to MySQL but not safe

sudo tune2fs -o journal_data_writeback /dev/sda1 (use the right partition)

vim /etc/fstab

ext4 errors=remount-ro,noatime,nodiratime,data=writeback,barrier=0,nobh,commit=100,nouser_xattr 0 1

vim /etc/my.cnf

innodb_flush_log_at_trx_commit = 2

Script for optimizing images in a directory (recursive) in Linux


#!/bin/sh

# script for optimizing images in a directory (recursive)
# pngcrush & jpegtran settings from:
# http://developer.yahoo.com/performance/rules.html#opt_images

# pngcrush
for png in `find $1 -iname "*.png"`; do
echo "crushing $png ..."
pngcrush -rem alla -reduce -brute "$png" temp.png

# preserve original on error
if [ $? = 0 ]; then
mv -f temp.png $png
else
rm temp.png
fi
done

# jpegtran
for jpg in `find $1 -iname "*.jpg"`; do
echo "crushing $jpg ..."
jpegtran -copy none -optimize -perfect "$jpg" > temp.jpg

# preserve original on error
if [ $? = 0 ]; then
mv -f temp.jpg $jpg
else
rm temp.jpg
fi
done

The first post on Juzhax.com

The first post on Juzhax.com

My previous personal blog, Justin.my isn’t a personal blog anymore. It is most like a game walkthrough, solution sites more than a personal blog. And there is no place to let me share some technology related post there.

What am I going to share in here

I would like to share a lot of technology in here and I could blog down some linux command or some raspberry pi related testing, then will be some Arduino related article in here too. Sometimes I will share some of my personal photos in here too. I will update the blog in here often after this Chinese New Year. Happy Chinese New Year to everyone visit in here.