Batch rename files in Bash

$ rename s/"SEARCH"/"REPLACE"/g *

This will replace the string SEARCH with REPLACE in every file (that is, *). The /g means global, so if you had a "SEARCH SEARCH.jpg", it would be renamed "REPLACE REPLACE.jpg". If you didn't have /g, it would have only done substitution once, and thus now named "REPLACE SEARCH.jpg". If you want case insensitive, add /i (that would be, /gi or /ig at the end).

With regular expressions, you can do lots more. For example, if you want to append something to every file: $ rename s/'^'/'MyPrefix'/ * That would add MyPrefix to the beginning of every filename. You can also do ending: $ rename s/'$'/'MySuffix'/ *

Also, the -n option will just show what would be renamed, then exit. This is useful, because you can make sure you have your command right before messing all your filenames up. :)

How does one write code that best utilizes the CPU cache to improve performance?

The cache is there to reduce the number of times the CPU would stall waiting for a memory request to be fulfilled (avoiding the memory latency), and as a second effect, possibly to reduce the overall amount of data that needs to be transfered (preserving memory bandwidth). Techniques for avoiding suffering from memory fetch latency is typically the first thing to consider, and sometimes helps a long way. The limited memory bandwidth is also a limiting factor, particularly for multicores and multithreaded applications where many threads wants to use the memory bus. A different set of techniques help addressing the latter issue. Improving spatial locality means that you ensure that each cache line is used in full once it has been mapped to a cache. When we have looked at various standard benchmarks, we have seen that a surprising large fraction of those fail to use 100% of the fetched cache lines before the cache lines are evicted. Improving cache line utilization helps in three respects: It tends to fit more useful data in the cache, essentially increasing the effective cache size. It tends to fit more useful data in the same cache line, increasing the likelyhood that requested data can be found in the cache. It reduces the memory bandwidth requirements, as there will be fewer fetches. Common techniques are: Use smaller data types Organize your data to avoid alignment holes (sorting your struct members by decreasing size is one way) Beware of the standard dynamic memory allocator, which may introduce holes and spread your data around in memory as it warms up. Make sure all adjacent data is actually used in the hot loops. Otherwise, consider breaking up data structures into hot and cold components, so that the hot loops use hot data. avoid algorithms and datastructures that exhibit irregular access patterns, and favor linear datastructures. We should also note that there are other ways to hide memory latency than using caches. Modern CPU:s often have one or more hardware prefetchers. They train on the misses in a cache and try to spot regularities. For instance, after a few misses to subsequent cache lines, the hw prefetcher will start fetching cache lines into the cache, anticipating the application's needs. If you have a regular access pattern, the hardware prefetcher is usually doing a very good job. And if your program doesn't display regular access patterns, you may improve things by adding prefetch instructions yourself. Regrouping instructions in such a way that those that always miss in the cache occur close to each other, the CPU can sometimes overlap these fetches so that the application only sustain one latency hit (Memory level parallelism). To reduce the overall memory bus pressure, you have to start addressing what is called temporal locality. This means that you have to reuse data while it still hasn't been evicted from the cache. Merging loops that touch the same data (loop fusion), and employing rewriting techniques known as tilingor blocking all strive to avoid those extra memory fetches. While there are some rules of thumb for this rewrite exercise, you typically have to carefully consider loop carried data dependencies, to ensure that you don't affect the semantics of the program. These things are what really pays off in the multicore world, where you typically wont see much of throughput improvements after adding the second thread. share|edit answered Jun 19 '09 at 16:20 Mats N 43143

+1, this is excellent, thank you. – Antony Vennard Dec 29 '10 at 22:10

up vote22down vote

I recommend reading the 9-part article What every programmer should know about memory by Ulrich Drepper if you're interested in how memory and software interact. It's also available as a 104-page PDF. Sections especially relevant to this question might be Part 2 (CPU caches) and Part 5 (What programmers can do - cache optimization). share|edit answered Apr 18 '09 at 12:56 Tomi Kyöstilä 60358

Python Time Conversion

From http://emilics.com/blog/article/python_time.html

Python Time Conversion

In python, there are four types that are commonly used to manage time. Timestamps, time tuples, datetime objects, and strings. Programmers often have to convert between these types depending on the situation or API. This article will describe all conversion patterns.

Python Time Conversion Table

Table 1 shows the python conversion table. Click on the pattern you want to see.

Table 1. python time conversion table

input \ output	datetime	time tuple	time stamp	string
datetime	—	datetime ↓ time tuple	datetime ↓ time stamp	datetime ↓ string
time tuple	time tuple ↓ datetime	—	time tuple ↓ time stamp	time tuple ↓ string
time stamp	time stamp ↓ datetime	time stamp ↓ time tuple	—	time stamp ↓ string
string	string ↓ datetime	string ↓ time tuple	string ↓ time stamp	—

datetime → time tuple

>>> dt = datetime.datetime(2010, 12, 31, 23, 59, 59)
>>> tt = dt.timetuple()
>>> print tt
time.struct_time(tm_year=2010, tm_mon=12, tm_mday=31, tm_hour=23, tm_min=59, tm_sec=59, ...)

datetime → time tuple

datetime → time stamp

>>> dt = datetime.datetime(2010, 12, 31, 23, 59, 59)
>>> ts = time.mktime(dt.timetuple())
>>> print ts
1293868799.0

datetime → time stamp

datetime → string

>>> dt = datetime.datetime(2010, 12, 31, 23, 59, 59)
>>> st = dt.strftime('%Y-%m-%d %H:%M:%S')
>>> print st
2010-12-31 23:59:59

datetime → string

time tuple → datetime

>>> tt = (2010, 12, 31, 23, 59, 59, 4, 365, 0)
>>> dt = datetime.datetime(tt[0], tt[1], tt[2], tt[3], tt[4], tt[5])
>>> print dt
2010-12-31 23:59:59
>>>
>>> dt = datetime.datetime(*tt[0:6])  # same with the code above
>>> print dt
2010-12-31 23:59:59

time tuple → datetime

time tuple → time stamp

>>> tt = (2010, 12, 31, 23, 59, 59, 4, 365, 0)
>>> ts = time.mktime(tt)
>>> print ts
1293868799.0

time tuple → time stamp

time tuple → string

>>> tt = (2010, 12, 31, 23, 59, 59, 4, 365, 0)
>>> st = time.strftime('%Y-%m-%d %H:%M:%S', tt)
>>> print st
2010-12-31 23:59:59

time tuple → string

time stamp → datetime

>>> ts = 1293868799.0
>>> dt = datetime.datetime.fromtimestamp(ts)     # for local time
>>> print dt
2010-12-31 23:59:59
>>>
>>> dt = datetime.datetime.utcfromtimestamp(ts)  # for UTC
>>> print dt
2011-01-01 07:59:59

time stamp → datetime

time stamp → time tuple

>>> ts = 1293868799.0
>>> tt = time.localtime(ts)
>>> print tt
time.struct_time(tm_year=2010, tm_mon=12, tm_mday=31, tm_hour=23, tm_min=59, tm_sec=59, ...)
>>>
>>> tt = time.gmtime(ts)
>>> print tt
time.struct_time(tm_year=2011, tm_mon=1, tm_mday=1, tm_hour=7, tm_min=59, tm_sec=59, ...)

time stamp → time tuple

time stamp → string

>>> ts = 1293868799.0
>>> st = datetime.datetime.fromtimestamp(ts).strftime('%Y-%m-%d %H:%M:%S')
>>> print st
2010-12-31 23:59:59
>>>
>>> st = datetime.datetime.utcfromtimestamp(ts).strftime('%Y-%m-%d %H:%M:%S')
>>> print st
2011-01-01 07:59:59

time stamp → string

string → datetime

>>> s = '2010-12-31 23:59:59'
>>> dt = datetime.datetime.strptime(s, '%Y-%m-%d %H:%M:%S')
>>> print dt
2010-12-31 23:59:59

string → datetime

string → time tuple

>>> st = '2010-12-31 23:59:59'
>>> tt = time.strptime(st, '%Y-%m-%d %H:%M:%S')
>>> print tt
time.struct_time(tm_year=2010, tm_mon=12, tm_mday=31, tm_hour=23, tm_min=59, tm_sec=59, ...)

string → time tuple

string → time stamp

>>> s = '2010-12-31 23:59:59'
>>> ts = time.mktime(time.strptime(s, '%Y-%m-%d %H:%M:%S'))
>>> print ts
1293868799.0

string → time stamp

Install Emacs 24.1 on Ubuntu 12.04

From
http://royontechnology.blogspot.hk/2012/06/installing-emacs-241.html

Though we can directly install from the Ubuntu Software Center, but it is more flexible to build it.
Installing Emacs 24.1
I was excited to see the announcement that Emacs 24.1 has been released.  I wanted to install and try. There were a few glitches that I faced during the installation. I thought it might be useful for someone who might be hitting the same blocks.

Once I downloaded the source code and verified the signature, I unzipped the Emacs 24.1 source code. The first "configure" run gave the following error:
configure: error: You seem to be running X, but no X development libraries
were found.  You should install the relevant development files for X
and for the toolkit you want, such as Gtk+, Lesstif or Motif.  Also make
sure you have development files for image handling, i.e.
tiff, gif, jpeg, png and xpm.
If you are sure you want Emacs compiled without X window support, pass
  --without-x
to configure.
To address this error I had to do the following:
sudo apt-get install libgtk2.0-dev libtiff4-dev libgif-dev libjpeg62-dev libpng12-dev libxpm-dev
Then when I attempted to run configure again, I got the following error:
configure: error: The required function `tputs' was not found in any library.
These libraries were tried: libncurses, libterminfo, libtermcap, libcurses.
Please try installing whichever of these libraries is most appropriate
for your system, together with its header files.
For example, a libncurses-dev(el) or similar package.
To resolve this error I had to do the following:
sudo apt-get install libncurses-dev
Thats it. The installation was smooth. Here is the summary of commands you need to run:
sudo apt-get install libgtk2.0-dev libtiff4-dev libgif-dev libjpeg62-dev libpng12-dev libxpm-dev libncurses-dev
#Note, libjpeg-dev, libpng12-dev's version may be different.
# You can skip this step if you don't want to verify the signature.
gpg --verify emacs-24.1.tar.bz2.sig emacs-24.1.tar.bz2
tar xvfj emacs-24.1.tar.bz2
cd emacs-24.1
./configure
make
sudo make install
Posted by Roy at 4:25:00 PM
Labels: Emacs

Install scipy for Ubuntu 12.04

Download from git
https://github.com/scipy/scipy

1. Install fortran compiler. e.g. gfortran.
2. Install blas, lapack (search libblas, liblapack on the Ubuntu Software Center)

3. $ python setup.py install --prefix=$MYDIR ($MYDIR can be /usr/local)

using algorithm in latex

[Latex] 如何在Latex上寫algorithm

[转]

作者： Wei Chung Cheng 發佈於 上午1:57 2011年1月21日星期五

做個紀錄吧，我下關鍵字都找不到清晰的教學XD

希望以後有人要用到的時候比較好查！



首先你需要灌一些package，灌package的方式，我以MiKTeX為例子。

簡單的來說，開始 -> MiKTeX -> Maintenance (Admin) -> Package Manager (Admin)

點進去之後，他會搜尋所有的Package。

然後你點選你需要的Package 再按那個"+"的按鈕就會更新package了！

通常Algorithm 會用到哪些package呢？

你可以灌以下這些Package：
algorithms, algorithm2e, program, alg, algorithmicx, pseudocode

安裝完成之後，回到你的tex檔。

在\begin{document}之前，加入以下這兩行：
\usepackage{algorithmic} - 對應algorithmicx

\usepackage{algorithm} - 對應algorithms

因為我目前只用到這兩個，相對應其他package的usepackage 稍微搜尋一下就會有了！

有時候加完之後complie 會出現algorithm.sty找不到的訊息。

這時候就手動將algorithm.sty放入你目前的資料夾中就可以了！

至於去哪找到algorithm.sty呢？

資料夾 "MiKTeX 2.8\tex\latex\algorithms" 底下就找得到。

另外附上簡單的Sample

\algsetup{indent=2em}

\renewcommand{\algorithmicrequire}{\textbf{Input:}}

\renewcommand{\algorithmicensure}{\textbf{Output:}}

\newcommand{\factorial}{\ensuremath{\mbox{\sc Factorial}}}

\begin{algorithm}[h!]

\caption{$\factorial(n)$}\label{alg:factorial}

\begin{algorithmic}[1]

\REQUIRE An integer $n \geq 0$.

\ENSURE The value of $n!$.

\medskip

\IF {$n = 0$}

\RETURN $1$

\ELSE

\RETURN $n \cdot \factorial(n-1)$

\ENDIF

\end{algorithmic}

\end{algorithm

前兩行renewcommand 是：
將require以Input: 形式表示及ensure以Output:形式表示。

網路上範例一：
\begin{algorithm}[htb]
  \caption{ Framework of ensemble learning for our system.}
  \label{alg:Framwork}
  \begin{algorithmic}[1]
    \Require
      The set of positive samples for current batch, $P_n$;
      The set of unlabelled samples for current batch, $U_n$;
      Ensemble of classifiers on former batches, $E_{n-1}$;
    \Ensure
      Ensemble of classifiers on the current batch, $E_n$;
    \State Extracting the set of reliable negative and/or positive samples $T_n$ from $U_n$ with help of $P_n$;
    \label{code:fram:extract}
    \State Training ensemble of classifiers $E$ on $T_n \cup P_n$, with help of data in former batches;
    \label{code:fram:trainbase}
    \State $E_n=E_{n-1}cup E$;
    \label{code:fram:add}
    \State Classifying samples in $U_n-T_n$ by $E_n$;
    \label{code:fram:classify}
    \State Deleting some weak classifiers in $E_n$ so as to keep the capacity of $E_n$;
    \label{code:fram:select} \\
    \Return $E_n$;
  \end{algorithmic}
\end{algorithm}

排版效果圖：

網路上範例二：

\begin{algorithm}[h]
  \caption{An example for format For \& While Loop in Algorithm}
  \begin{algorithmic}[1]
    \For{each $i\in [1,9]$}
      \State initialize a tree $T_{i}$ with only a leaf (the root);
      \State $T=T\cup T_{i};$
    \EndFor
    \ForAll {$c$ such that $c\in RecentMBatch(E_{n-1})$}
      \label{code:TrainBase:getc}
      \State $T=T\cup PosSample(c)$;
      \label{code:TrainBase:pos}
    \EndFor;
    \For{$i=1$; $i<n$; $i++$ }
      \State $//$ Your source here;
    \EndFor
    \For{$i=1$ to $n$}
      \State $//$ Your source here;
    \EndFor
    \State $//$ Reusing recent base classifiers.
    \label{code:recentStart}
    \While {$(|E_n| \leq L_1 )and( D \neq \phi)$}
      \State Selecting the most recent classifier $c_i$ from $D$;
      \State $D=D-c_i$;
      \State $E_n=E_n+c_i$;
    \EndWhile
    \label{code:recentEnd}
  \end{algorithmic}
\end{algorithm}

排版效果圖：

個人範例：

\begin{algorithm}[h]
  \caption{Conjugate Gradient Algorithm with Dynamic Step-Size Control}
  \label{alg::conjugateGradient}
  \begin{algorithmic}[1]
    \Require
      $f(x)$: objective funtion;
      $x_0$: initial solution;
      $s$: step size;
    \Ensure
      optimal $x^{*}$    
    \State initial $g_0=0$ and $d_0=0$;
    \Repeat
      \State compute gradient directions $g_k=\bigtriangledown f(x_k)$;
      \State compute Polak-Ribiere parameter $\beta_k=\frac{g_k^{T}(g_k-g_{k-1})}{\parallel g_{k-1} \parallel^{2}}$;
      \State compute the conjugate directions $d_k=-g_k+\beta_k d_{k-1}$;
      \State compute the step size $\alpha_k=s/\parallel d_k \parallel_{2}$;
    \Until{($f(x_k)>f(x_{k-1})$)}
  \end{algorithmic}
\end{algorithm}

排版效果圖：
  

先前所使用的套件為algorithm或algorithmic

接下來介紹另一個寫algorithm的套件alogrithm2e

首先使用\usepackage指令
\usepackage[linesnumbered,boxed]{algorithm2e}

接下來是網路範例：
\begin{algorithm}
  \caption{identifyRowContext}
  \KwIn{$r_i$, $Backgrd(T_i)$=${T_1,T_2,\ldots ,T_n}$ and similarity threshold $\theta_r$}
  \KwOut{$con(r_i)$}
  $con(r_i)= \Phi$\;
  \For{$j=1;j \le n;j \ne i$}
  {
    float $maxSim=0$\;
    $r^{maxSim}=null$\;
    \While{not end of $T_j$}
    {
      compute Jaro($r_i,r_m$)($r_m\in T_j$)\;
      \If{$(Jaro(r_i,r_m) \ge \theta_r)\wedge (Jaro(r_i,r_m)\ge r^{maxSim})$}
      {
        replace $r^{maxSim}$ with $r_m$\;
      }
    }
    $con(r_i)=con(r_i)\cup {r^{maxSim}}$\;
  }
  return $con(r_i)$\;
\end{algorithm}

排版效果圖：
  

延伸幾個問題：

一、如何修改Algorithm的標題為中文的"演算法”？

在\begin{document}之前加入\renewcommand{\algorithmcfname}{算法} 即可(註：需先安裝中文字形)

二、如何去掉演算法中的豎線？

加入 \SetAlgoNoLine 指令在\begin{algorithm}之後

排版效果圖：

  

三、還可以使用其他標題樣式？

也可以使用\usepackage[ruled,vlined]{algorithm2e}

 排版效果圖：

  

其他樣式可以參考：http://mirror.bjtu.edu.cn/CTAN/macros/latex/contrib/algorithm2e/algorithm2e.pdf

關於algorithm2e還有以下一些information
The algorithm2e LaTeX package conflicts with several others over the use of the algorithm identifier.  

A common indicator is something like this message: Too many }'s.l.1616     }

To resolve the issues, simply put the following just before the inclusion of the algorithm2e package:

\makeatletter
\newif\if@restonecol
\makeatother
\let\algorithm\relax
\let\endalgorithm\relax

Configure sMobileNet of HKUST on Linux | Linux and Virtualization

Configure sMobileNet of HKUST on Linux | Linux and Virtualization

The wireless network connection configuration for sMobileNet in NetworkManager

The SSID is “sMobileNet”

Wireless security is “WPA & WPA2 Enterprise”

Authentication is “Protected EAP (PEAP)”

Anonymous identity is “” (empty)

CA certificate is “(None)” (there will be a warning appear, we can ignore it)

PEAP version is “Automatic”

Inner authentication is “MSCHAPv2″

Username is “username@ust.hk”

Password is your password.

A sample security configuration