PostgreSQL – Copy To and Copy From using Select queries

When you need to copy a small set of data from one database to an another (say, from dev system to a QA system), you could use the copy feature of PSQL.

To get the data in CSV of a select query:

duggout=> copy (select * from lookups where lkp_type = 'homepage_items' ) to STDOUT with csv;
6514,homepage_items,side_1,6,,,,,,,,,2014-10-06 15:47:25.216133,2014-10-06 15:47:25.216133
6515,homepage_items,list_main,"1,2,3,4,5,6,7",,,,,,,,,2014-10-08 02:52:52.995771,2014-10-08 02:52:52.995771
6516,homepage_items,image_main,"5,6,7",international_match,f,,,,,,,2014-10-05 14:06:55.591311,2014-10-11 08:14:47.196494

To load the data into the table in another environment:

COPY lookups FROM STDIN with csv;

Now, paste the CSV data with a trailing new line. And then \. (backslash with a dot). This should append the CSV data in the target table. When you want to copy more data, use the file option instead of STDIN/STDOUT. Check the doc for the usages.

To copy a large table, use the binary option:

remote_db=# copy table_name to '/tmp/a_path' with binary;
COPY 38765

You could copy the file to your local using scp, if the file is in a remote server. Then,

local_db=# copy table_name from 'local_path' with binary;
COPY 38765

It is as simple as that!

Rails – Redirect for incorrect URL slug, like StackOverflow

When you visit a StackOverflow question with a different URL slug, they redirect you to the right URL with the correct slug. This is important for two reasons:

  1. When the title of the question changes, the URL with the old title will be obsolete
  2. Keeps a single Permalink for every question

In your controller:

before_action :redirect_if_incorrect_url_code, only: [:show]

private
def redirect_if_incorrect_url_code
return if @model.url_code == params[:url_code]
redirect_to @model.route_param
end

In your routes:

get 'models/:id(/:url_code)', to: 'models#show', constraints: {url_code: /[a-z\-]+/, id: /\d+/}, as: :model

In your model:


def route_param
[id: id, url_code: url_code][0]
end

Showing a spinner/loading cursor icon for Rails 4 Turbolink

In Rails 4, Turbolinks loads pages using AJAX. It fetches only the body content of the new page and updates the current page with the new content and the title as well. This avoids reloading of all the assets such as Javascript and CSS. This has one small drawback though. The user might not really know if the page is loading or not, since there are no indication in the tab. To fix this, we can update the cursor of the page to a loading cursor whenever turbolinks starts to load a page. Create a new coffee script file with the following content, and load it after the turbolinks is loaded.


$ =>
startSpinner = ->
  $("html").css "cursor", "progress"
  return
stopSpinner = ->
  $("html").css "cursor", "auto"
  return
$(document).on "page:fetch", startSpinner
$(document).on "page:receive", stopSpinner

The following code lets you trigger a page tracking for Piwik:


$(document).on 'page:change', ->
if window._gaq?
_gaq.push ['_trackPageview']
else if window.pageTracker?
pageTracker._trackPageview()
return

Discourse Installation (via docker) on existing Apache server

If you follow the official install guide of Discource, on a existing server that hosts a Rails Application on Apache/Passenger, you will get into  a PORT issue. Apache is responding to 80 already and you want Discourse (inside docker) to respond to 80 as well. So you will be required to run the Discourse site on a non-standard port such as 85.

In containers/app.yml (from /var/docker), do:

expose:
- "85:80" # fwd host port 85 to container port 80 (http)

Now you can access you Discourse site on http://forum.example.com:85

So far, so good. But, having a port in your URL looks dirty. You need a way to hide the port from the user. Apache’s proxy module comes to the rescue. Create a site configuration named forum.example.com with the the following content:

<VirtualHost *:80>
ProxyPreserveHost On
ProxyRequests Off
ServerName forum.example.com
ProxyPass / http://localhost:85/
ProxyPassReverse / http://localhost:85/
</VirtualHost>

Enable the site and restart Apache. Now, whenever you visit forum.example.com, Apache will proxy the request to forum.example.com:85 automatically.

Pass arguments to Linux alias command

Every now and then, I write a command to see if a process is running. To alias this, I need to pass the search string to the alias as below:

Add this in .bash_profile from your home directory:

function check() { ps -eaf | grep "$@" ;}

Then, you could use the check command like this:

~ $ check rails
501 7139 6302 0 10:24AM ttys006 0:03.12 /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby bin/rails s
501 9913 8580 0 2:16PM ttys007 0:00.00 grep rails
~ $

List of POS Tags in NLTK python

~ $ python
Python 2.7.2 (default, Oct 11 2012, 20:14:37)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import nltk
>>> nltk.help.upenn_tagset()
$: dollar
$ -$ --$ A$ C$ HK$ M$ NZ$ S$ U.S.$ US$
'': closing quotation mark
' ''
(: opening parenthesis
( [ {
): closing parenthesis
) ] }
,: comma
,
--: dash
--
.: sentence terminator
. ! ?
:: colon or ellipsis
: ; ...
CC: conjunction, coordinating
& 'n and both but either et for less minus neither nor or plus so
therefore times v. versus vs. whether yet
CD: numeral, cardinal
mid-1890 nine-thirty forty-two one-tenth ten million 0.5 one forty-
seven 1987 twenty '79 zero two 78-degrees eighty-four IX '60s .025
fifteen 271,124 dozen quintillion DM2,000 ...
DT: determiner
all an another any both del each either every half la many much nary
neither no some such that the them these this those
EX: existential there
there
FW: foreign word
gemeinschaft hund ich jeux habeas Haementeria Herr K'ang-si vous
lutihaw alai je jour objets salutaris fille quibusdam pas trop Monte
terram fiche oui corporis ...
IN: preposition or conjunction, subordinating
astride among uppon whether out inside pro despite on by throughout
below within for towards near behind atop around if like until below
next into if beside ...
JJ: adjective or numeral, ordinal
third ill-mannered pre-war regrettable oiled calamitous first separable
ectoplasmic battery-powered participatory fourth still-to-be-named
multilingual multi-disciplinary ...
JJR: adjective, comparative
bleaker braver breezier briefer brighter brisker broader bumper busier
calmer cheaper choosier cleaner clearer closer colder commoner costlier
cozier creamier crunchier cuter ...
JJS: adjective, superlative
calmest cheapest choicest classiest cleanest clearest closest commonest
corniest costliest crassest creepiest crudest cutest darkest deadliest
dearest deepest densest dinkiest ...
LS: list item marker
A A. B B. C C. D E F First G H I J K One SP-44001 SP-44002 SP-44005
SP-44007 Second Third Three Two * a b c d first five four one six three
two
MD: modal auxiliary
can cannot could couldn't dare may might must need ought shall should
shouldn't will would
NN: noun, common, singular or mass
common-carrier cabbage knuckle-duster Casino afghan shed thermostat
investment slide humour falloff slick wind hyena override subhumanity
machinist ...
NNP: noun, proper, singular
Motown Venneboerger Czestochwa Ranzer Conchita Trumplane Christos
Oceanside Escobar Kreisler Sawyer Cougar Yvette Ervin ODI Darryl CTCA
Shannon A.K.C. Meltex Liverpool ...
NNPS: noun, proper, plural
Americans Americas Amharas Amityvilles Amusements Anarcho-Syndicalists
Andalusians Andes Andruses Angels Animals Anthony Antilles Antiques
Apache Apaches Apocrypha ...
NNS: noun, common, plural
undergraduates scotches bric-a-brac products bodyguards facets coasts
divestitures storehouses designs clubs fragrances averages
subjectivists apprehensions muses factory-jobs ...
PDT: pre-determiner
all both half many quite such sure this
POS: genitive marker
' 's
PRP: pronoun, personal
hers herself him himself hisself it itself me myself one oneself ours
ourselves ownself self she thee theirs them themselves they thou thy us
PRP$: pronoun, possessive
her his mine my our ours their thy your
RB: adverb
occasionally unabatingly maddeningly adventurously professedly
stirringly prominently technologically magisterially predominately
swiftly fiscally pitilessly ...
RBR: adverb, comparative
further gloomier grander graver greater grimmer harder harsher
healthier heavier higher however larger later leaner lengthier less-
perfectly lesser lonelier longer louder lower more ...
RBS: adverb, superlative
best biggest bluntest earliest farthest first furthest hardest
heartiest highest largest least less most nearest second tightest worst
RP: particle
aboard about across along apart around aside at away back before behind
by crop down ever fast for forth from go high i.e. in into just later
low more off on open out over per pie raising start teeth that through
under unto up up-pp upon whole with you
SYM: symbol
% & ' '' ''. ) ). * + ,. < = > @ A[fj] U.S U.S.S.R * ** ***
TO: "to" as preposition or infinitive marker
to
UH: interjection
Goodbye Goody Gosh Wow Jeepers Jee-sus Hubba Hey Kee-reist Oops amen
huh howdy uh dammit whammo shucks heck anyways whodunnit honey golly
man baby diddle hush sonuvabitch ...
VB: verb, base form
ask assemble assess assign assume atone attention avoid bake balkanize
bank begin behold believe bend benefit bevel beware bless boil bomb
boost brace break bring broil brush build ...
VBD: verb, past tense
dipped pleaded swiped regummed soaked tidied convened halted registered
cushioned exacted snubbed strode aimed adopted belied figgered
speculated wore appreciated contemplated ...
VBG: verb, present participle or gerund
telegraphing stirring focusing angering judging stalling lactating
hankerin' alleging veering capping approaching traveling besieging
encrypting interrupting erasing wincing ...
VBN: verb, past participle
multihulled dilapidated aerosolized chaired languished panelized used
experimented flourished imitated reunifed factored condensed sheared
unsettled primed dubbed desired ...
VBP: verb, present tense, not 3rd person singular
predominate wrap resort sue twist spill cure lengthen brush terminate
appear tend stray glisten obtain comprise detest tease attract
emphasize mold postpone sever return wag ...
VBZ: verb, present tense, 3rd person singular
bases reconstructs marks mixes displeases seals carps weaves snatches
slumps stretches authorizes smolders pictures emerges stockpiles
seduces fizzes uses bolsters slaps speaks pleads ...
WDT: WH-determiner
that what whatever which whichever
WP: WH-pronoun
that what whatever whatsoever which who whom whosoever
WP$: WH-pronoun, possessive
whose
WRB: Wh-adverb
how however whence whenever where whereby whereever wherein whereof why
``: opening quotation mark
` ``

Preventing NullPointerException in tMap in Talend

When you have complex expressions inside a tMap, there could be many places that result in NullPointerException. Instead of debugging individual lines, open configuration of tMap and uncheck ‘Die on Error’. This will add a ErrorReject output for this tMap. Map this ErrorReject to a file and see the error message for the rejected records.

Talend tMap - Die on Error option
Talend tMap – Die on Error option

Reference – http://www.talendforge.org/forum/viewtopic.php?id=26668