Sunday, 12 September 2021

Modules in Node.js

Node.js uses CommonJS module format. It uses require to include a module. It uses module.exports to expose a modules. The npm ecosystem also uses this format.

Use require to include a module. The below we include express module.


const express = require('express')
const app = express()

app.get('/', (req, res) => {
  res.send('Hello World!')
})

const port = 2000
app.listen(port, () => {
  console.log(`listen to port 2000...`)
})

Use exports to let other codes to use this module. For example, AWS Lambda function can be written in Node.js

exports.myFunction = (event, context, callback) => {
        //do something
}

The below is a hello world module and how to use it

//in hello-world.js create a module
module.exports = name => console.log("hello to " + name)

//in main.js
var hw = require('./hello-world')
hw("kei")

//run it
node main.js //output: hello to kei
//another example
//in config.js
module.exports = {
  dbPort : 8080,
  dbHost : "etc",
  dbName : "nom",
}

//in main.js
var config = require("./config")
console.log("config:", config);
//more example
//in config
module.exports = {
  dbPort : 8080,
  dbHost : "etc",
  dbName : "nom",
  hello: (name)=> "hello to " + name
}

//in main
var leo = require("./config")

console.log(leo)

console.log(leo.hello("smith"))

//output node main.js
{
  dbPort: 8080,
  dbHost: 'etc',
  dbName: 'nom',
  hello: [Function: hello]
}
hello to smith

However, for ES6, it uses another format. It uses export and import to export and include a module. Here is post about default values in ES6 module

Monday, 6 September 2021

Java Queue

Both PriorityQueue and LinkedList implement Java Queue interface.

There are these methods. Every is a pair. The first one will throw error on failure. The second will return special value instead of throwing error.

  • add(e) offer(e)
  • remove() poll()
  • element() peek()

For LinkedList, we can call removeFirst()

Java PriorityQueue

java PriorityQueue implements java.util.AbstractQueue class which extends java.util.Collection

The head of the queue is the least element with respect to the specified ordering. However, can provide Collections.reverseOrder() in the constructor to create a max heap.

useful methods:

  • peek()
  • poll()

methods from java.util.Collection

  • size()
  • add()
  • isEmpty()

Here is a Leetcode problem leetcode 1046 which can be solved by max heap


import java.util.*;
class Solution {
    public int lastStoneWeight(int[] stones) {
        //create a max heap. The head has the max value
        var q = new PriorityQueue<Integer>(Collections.reverseOrder());

        //add element into heap
        for (int s: stones) {
            q.add(s);
        }

        while(q.size()>1) {
            int x = q.poll();
            int y = q.poll();

            if (x != y) {
            	//put result stone back into queue
                q.add(Math.abs(x-y));
            }
		}

        //check q.size
        if (q.size()==1) {
            return q.poll();
        } else {
            return 0;
        }
    }
}

Tree Level Traversal

The below is a solution for LeetCode 104

public int maxDepth(TreeNode root) {
        if (root ==null) return 0;
        
        var myq = new LinkedList<TreeNode>();
        myq.add(root);
        var ans = 0;
        
        while(!myq.isEmpty()) {
            //queue only holde nodes of current level
            var size = myq.size();
            
            //every iteration is one level because of size
            for(int i=0; i<size; i++) {
                TreeNode cur = myq.poll();
                if (cur.left!=null) myq.add(cur.left);
                if (cur.right!=null) myq.add(cur.right);
             }
            
            ans++;
        }
        
        return ans;
        
    }

Use Paire as queue element

2462

Priority Queue using custom comparator

Leetcode 2099

class Solution {

  public int[] maxSubsequence(int[] nums, int k) {
    var pq = new PriorityQueue<int[]>((a, b) -> a[1] - b[1]);

    for (int i = 0; i < nums.length; i++) {
      int[] t = { i, nums[i] };
      if (pq.size() < k) {
        pq.add(t);
      } else {
        var e = pq.peek();
        if (e[1] < nums[i]) {
          pq.poll();
          pq.add(t);
        }
      }
    }

    int[] index = new int[k];
    int j = 0;
    while (pq.size() > 0) {
      var e = pq.poll();

      index[j] = e[0];
      j++;
    }

    Arrays.sort(index);

    int[] ans = new int[k];
    j = 0;
    for (int i : index) {
      ans[j] = nums[i];
      j++;
    }

    return ans;
  }
}

array as queue element

//See Leetcode 373
PriorityQueue<int[]> minHeap = new PriorityQueue<> ((a, b) -> (a[0] + a[1]) - (b[0] + b[1]));
       
//init. the last two elements are index for two array
minHeap.add(new int[]{nums1[0], nums2[0], 0 ,0});

How to inject codes into blogger

Basically you need to add highlight.js into your blogger template. Then wrapper your code. See highlight.js usage

Here is a good tutorial about that: https://www.youtube.com/watch?v=APNWCe56cQA

Create REST API by AWS API Gateway and AWS Lambda

 We can create a REST API using AWS API Gateway and AWS Lambda by Proxy integration.

For Proxy integration, when API Gateway receives the request, it sends proxy resources such as path information and request body to Lambda function. Lambda function generates a responses based on information sent by API gateway. This response is sent back to API Gateway which send this response back to API user. In Lambda function, you can call other aws services such as store data in s3 bucket and send message to sqs before send back the response.

Step 1: Create a AWS Lamda function

Use Python as example. After create function, need to edit lambda_hander

 
import json

def lambda_handler(event, context):
    # event has info of request passed by API Gateway
    userId = event['queryStringParameters']['userId']

    # build response body
    body = {"user_id": userId, "message": "processed by Lambda"}

    # build response
    response = {'statusCode': 200, 'headers': {}}

    # let API caller know that our body is in json format
    response['headers']['content-type'] = 'application/json'
    response['body'] = json.dumps(body)

    return response

Note for node.js. In node.js, you can do

 
exports.myhandler = {event, context, callback) => {
        //get path parameters
        event.pathParamters...

        //get body
        event.body

        //get query string
        event.queryStringParameters ...

}

Note for debug:

For Python, use print something

For node.js, console.log something

Then we can see these debug info in cloudwatch


Step 2: Create a API Gateway

1. Pick Rest type

2. Create resource. Give it a name

3. Create method. We choose GET for this example.

4. In Get set up. Choose Integration type: Lambda Function. Also check "Use Lambda Proxy integration". Link the lambda function just created.


Step 3: deplay this API Gateway

Step 4: try it

find the api url and append a user id such as ?user_id=1

Step 5: add custom domain

You may want to add custom domain for this api.

Here is a good youtube video about setting up custom domain for API gateway

Here is a good youtube video about Gateway API and lamda function.

Friday, 27 August 2021

Put component, app, store together in React Redux

Redux store like a React state database. To display a component, we need retrieve states from store. When we take some actions on component, some states will be updated and we need to save these states in store. How do we connect all these pieces together? We will use Redux connect API!

Step one: define mapStateToProps function in component

To display a component, we need props. These props comes from states. We need a mapping between component's props and store's states. This function take state as parameter and return a mapping object.

const mapStateToProps = (state) => {
    return {
        user: state.app.user,
        name: getName(state),
        ....
    }
};

Noted we can do a direct mapping. Also we can call a function in reducer to do mapping.

These props can be used in function component directly

function myfuncComponent({user, name}) {
...
}

Also can access prop using the syntax below

 this.props.user

Also noted we can access props generated by step two in the same way.

There is another pattern to do mapStateToProps. Map some props to states by modify states. Then copy all left states as props. See below

const mapStateToProps = (state) => {
    const values = {
     user: state.app.user,
        name: getName(state),
        ....
    };
    //the first parameter is target. Here merge values 
    //with state. Then copy to the target and return target
    return Object.assign({}, state, values);
    
    //may do. only overwrite counut property and leave other properties intact
    return {...state, count:state.count+1}
};

Step two: define mapDispatchToProps function in component

This function uses dispatch as a paramter. It defines mapping between user action and redux action by dispatch user action. Reduct action will pass the action event to reducer. Reducer will change states. States will map props, and component display will be changed.

const mapDispatchToProps = (disptach, ownProps) => {
    return {
    	//when use handleSubmit, we can pass values
        handleSubmit: values => dispatch(saveValues(values))
        ...
    }
}

Example to use ownProps of mapDispatchToProps

const mapDispatchToProps = (dispatch, ownProps) => {
  return {
    toggleTodo: () => dispatch(toggleTodo(ownProps.todoId))
  }
}

Step 3: use connect api to create a container and export it

export default connect(
    mapStateToProps,
    mapDispatchToProps
)(ComponentName)

connect is a high order component. After call connect, it returns a new function. This new function will accept a component as argument and return an enchanced component. To understand this, we can do this in two step.

const newfunction = connect(mapStateToProps, mapDispatchToProps)
export default newfunction(ComponentName)

When write a component, we need some props and some event handlers. This is why we need to do state mapping and dispatch mapping.

Step 4: connect app, store and component. It is done in index.js of app

Use provider to provide store and put our container into provider. Our app will render provider in root element. By far, things are put together.

import {Provider} from 'react-redux'

import store from './redux/store'

//we export container as default in myComponent
import container from './myComponent'

//create a store. To create a store, you should 
//have written a reducer
import reducers from 'yourReducer'
import {createStore} from 'redux'

const store = createStore(reducers)

const root = document.getElementById('root')

ReactDOM.render(
        <Provider store={store}>
                <container />
        </Provider>, root
    );

Some nice explanation from stackoverflow

To see states of store

store.subscribe(() => console.log("state", store.getState()))

shouldComponentUpdate

Once I find that state is updated, but component is not updated. Eventually, I figured out that was caused by shouldComponentUpdate. The following prograph comes from React documentation.

Use shouldComponentUpdate() to let React know if a component’s output is not affected by the current change in state or props. The default behavior is to re-render on every state change, and in the vast majority of cases you should rely on the default behavior. shouldComponentUpdate() is invoked before rendering when new props or state are being received. Defaults to true. This method is not called for the initial render or when forceUpdate() is used.

Redux form

To create a redux form

//create a normal react component. 
//It can be a function component or class component
let myComponent = (props) => {
 return <form onSubmit={props.handleSubmit}> .... </form>
}

//convert to a redux-form component. 
//Noted that reduxFormCreator is a function
let reduxFormCreator = reduxForm({form: 'my-form'})
myComponent = reduxFormCreator(myComponent)

//we can combine the above two lines into one line
myComponent = reduxForm({form: 'my-form'})(myComponent)

//redux form provide a Field component. 
//Form elements like input or select needs to be replaced by Field
//this field component to make input to connect to redux store
<Field name="my_input_name" component="input" type="text" 
onChange={some function}/>
//another Field example
<Field name="group_by_session" onChange={this.updateAndFetch.bind(this, "group_by_session")}
 id="group_by_session" component="input" type="checkbox" />

The value from component prop of Field can be a stateless or stateful component. It can access these props

How to set intial values in Fields of Redux Form

submit form

//Redux-Form decorates your component with handleSubmit prop
let MyForm = ({
    handleSubmit, handleCloseRequestChangeModal, error, handleCloseRequestChangeModalError, formErrors
}) => {
    return (<>
        <Field
            name="storeName"
            label = "Store Name"
            component="input"
            validate={[formFieldValidator.required, formFieldValidator.notEmpty]}
        />
        <button className={"btn btn-success margin-right-3"} onClick={handleSubmit}>Save</button>
        </>)
}

When use this redux form, we need to pass a function for onSubmit

<MyForm  onSubmit={saveIntegration}/>

Add a hidden field in redux form and fill the hidden field dynamically

 <Field
name="recaptchaToken"
component="input"
type="hidden"
/>
handleRemoteSubmit: (token) => {
           dispatch(change('checkoutForm', 'recaptchaToken', token));
           dispatch(submit('checkoutForm'))
}

React function component

If a component only needs render function and props, we can write it in function component.

//create one
function Square(props) {
  return (
    <button className="square" onClick={props.onClick}>
      {props.value}
    </button>
  );
}

//to use it, we do NOT do a function call. Use it as normal component.
//pass two props. one is called value and the other is called onClick
 <Square
    value={this.state.squares[i]}
    onClick={() => this.handleClick(i)}
 />

Disptach

The Redux store has a method called dispatch. The only way to update the state is to call store.dispatch() and pass in an action object. The store will run its reducer function and save the new state value inside.

//here is how to apply this tech

//in the action.js, create a action creator
export const CHANGE_RESULTS_VIEW = 'CHANGE_RESULTS_VIEW';
export function changeResultsView(view) {
    return { type: CHANGE_RESULTS_VIEW, view: view}
}

//in container, import that action creator. When do mapDispatchToProps,
//use action creator to create a action, and this action is passed into
//dispatch. dispatch will change state of store using logic defined
//in reducer.
const mapDispatchToProps = (dispatch, ownProps) => {
    return {
       
        changeResultsView: view => dispatch(changeResultsView(view)),
        ...
        }
    }

in line conditional render

can use js logical && to add a element based on some condition

{books.length > 0 &&
	<h2>
		You have {books.length} books.
	</h2>
} 

Use update from immutability helpers

Update Library

Use update to keep the original object unchanged. Then create a upated new object. Here we use $set to set new value. Update has other commands such as $push

var update = require('immutability-helper');
 const obj = {a:5, b:3}
 const newObj = update(obj, {b: {$set:10}})
 console.log("new obj:", newObj)
 //newObj is {a:5, b:10}
 
 //the below is example for $merge. use to update multiple values
 update(state, {$merge:{details: '', showModal: false}})

Add inline css in jsx

properties with two names, like background-color, must be written with camel case syntax

<h1 style={{backgroundColor: "white"}}>Hello World!</h1>

//this css will keep spaces.jsx will remove space as default
<div style={{ whiteSpace: 'pre-wrap' }}>some text  <div>

Workflow to add event handler in react component using redux

step one: define a function in mapDispatchToProp

const mapDispatchToProps = (dispatch, ownProps) => {
    return {
        handleCloseDetailsModal: () => dispatch({ type: CLOSE_DETAILS_MODAL })
    }
};

step two: in render function, use object spread to get the function from props

step three: use that function in component. Note we put a function name. NOT call the function.

render() {
        const {handleCloseDetailsModal } = this.props;
        return (
            <div><DetailsModal
                isOpen={showModal}
                closeDetailsModal={handleCloseDetailsModal}
                details={details}
            /></div>
        );
    }

React hook and context

Good Explain

7 Hooks to know

Good tutorial

Create a context
//create a context object. Then create a function to return provider
const MyContext = createContext();
const GlobalContext = ({ children }) => {
	//define some state here
    var states = {...}
    
    //if it is class component, use this.props.children
    return (<MyContext.Provider values={...states}> {children} 
    </MyContext.Provider>)
}
To consume context

There are several different ways. By now the most popular is to use useContext

//here useGlobalContext has values and we can put this 
//line in any component. Therefore, this is just another way to share
//states between different components. Now useGlobalContext has values
//defined in the provider
  const useGlobalContext = useContext(Context)
import {useState} from 'react';

function TestB() {
    const [state, setState] = useState(0)

    return (
        <div>
            <p>{state}</p>
            <button onClick={() => setState(state + 1)}>More</button>
        </div>
    )
}

setState function in React.Component

We can use setState() to change the state of the component directly as well as through an arrow function.

//change directly by passing an object
setState({ stateName : updatedStateValue })

//use function to change state
setState((prevState) => ({ 
   stateName: prevState.stateName + 1 
}))

React Context

Introduction to React Context

React Hook Form

Good Tutorial

JS Cheating Sheet

Sheet

Class Component Benefit

export default class GooglePay extends React.Component {
    constructor(props) {
        super(props);

        //Google payment client
        this.paymentsClient = new google.payments.api.PaymentsClient({
            environment: props.cardInfo.googlePayEnv // 'TEST' Or 'PRODUCTION'
        });

        //request sent to Google
        this.paymentDataRequest = null;

        //need this method to create payment data request
        this.cardPaymentMethod = null;

        //create Google Pay Button
        this.createGooglePayButton();
    }

    state = {
        isLoading: false
    }
......

If parent component render multiple time and the child component is class component, the child component renders only once. However, if child component is function component, it will render the same time as parent component.

React and Bootsrap

import { Container, Row, Col } from "react-bootstrap";
return (
      <div className="App">
        <Container>
          <Row>
            <Col className="bg-primary p-2">1 of 2</Col>
            <Col className="bg-success p-2">2 of 2</Col>
          </Row>
          <Row>
            <Col className="bg-success p-2">1 of 3</Col>
            <Col className="bg-secondary p-2">2 of 3</Col>
            <Col className="bg-danger p-2">3 of 3</Col>
          </Row>
        </Container>
      </div>
    );


need to add bootstrap css

<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap@4.4.1/dist/css/bootstrap.min.css" integrity="sha384-Vkoo8x4CGsO3+Hhxv8T/Q5PaXtkKtu6ug5TOeNV6gBiFeWPGFN9MuhOf23Q9Ifjh" crossorigin="anonymous">

The below should work too. But I found it did not work. Have not figured out.

import 'bootstrap/dist/css/bootstrap.min.css';

React and TypeScript

Create a Project

Documentation

If you have created a folder already, do not put my_app and instead use "."

npx create-react-app . --template typescript

Access element in event hander

function MyButton() {
  function handleClick(event) {
    let button = event.target;
    alert(button.id);
  }
  return (
    <button id="leo" onClick={handleClick} className="my-button">
      Click Me
    </button>
  );
}

function component

//three ways to write function component
function Welcome({name, ...props}) {
    return <h1>Hello, {name}, yau are {props.age} old</h1>
}

function Welcome({name, age}) {
    return <h1>Hello, {name}, yau are {age} old</h1>
}

function Welcome(props) {
    return <h1>Hello, {props.name}, yau are {props.age} old</h1>
}

//use component
<Welcome name="smith" age="51" />

Tuesday, 24 August 2021

docker run or docker container run

They are exactly the same. docker container run is new syntax introduced in Docker 1.13.


Because there are so many docker commands, docker has put these commands in different groups. docker run is put into the group of docker container.


Here is a list of docker container command:


docker container commands


It is encouraged to use the new syntax. Therefore, let us do:


docker container run

Sunday, 1 August 2021

Import and export default values in JS modules

There should be only one default value in a module. There are two ways to export it.


export default function(n, m) {

    return n + m;

}   

another way


function add(n, m) {

    return n + m;

}   

export default add;

Assume the module name is math.js. To import it,


import sum from "./math.js";

Noted that we can give any name for default value when import it. Also do not need curly braces. For not default values, need curly braces.

For no default function export or import, need curly braces

import {mymethod} from '../../utilities/actions'
......
export {foo, awesom, check}

Import multiple functions from a module

import {get, set} as moduleName from 'module-name'
moduleName.get();

//or can do
import * as moduleName from 'module-name';
moduleName.get();

//or can do. But this may cause name collision
import {get,set} from 'module-name';
get()

//real life example
import * as yup from "yup";

username: yup.string().required("Username is required"),

(node:46383) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.

If get the above error when use node to run the script. The warning message you are encountering indicates that you are trying to load an ES (ECMAScript) module in a Node.js environment, but your current setup is not configured to recognize it as such.

node use commonjs module format

To solve it, one: Use "type": "module" in package.json. The alternative way is: rename your JavaScript files to use the .mjs extension instead of .js.

Saturday, 31 July 2021

ssh config file

ssh config file for proxy jump

When using ssh, you may need to remember lots of things such as host names, the sequence of servers to log into if you use proxy. However, you may use ssh config file to make life easy.

Here is an example. The file should be put in .ssh folder and named config

Host middle-server
StrictHostKeyChecking ask
HostName 123.35.678.12
User frank

Host target-server-name
ProxyJump middle-server
StrictHostKeyChecking ask
HostName 99.88.77.32

After create this config file, you can connect to target server by typing

ssh target-server-name

//also can use pem file
ssh -i NOC.pem ec2-user@target-server-name

Here is another example. It uses public and private key to log into jumpbox

host mm-jumpbox
HostName 123.213.58.123
User ec2-user
IdentityFile /Users/mark/.ssh/id_rsa

Host target-server-name
ProxyJump mm-jumpbox
StrictHostKeyChecking ask
HostName 10.123.44.44

Also can use pem file to log into jumpbox

Host mm-jumpbox
HostName 55.22.33.44
User ec2-user
IdentityFile ~/.ssh/NOC.pem

//then
ssh mm-jumpbox

ssh config for port forward

Assume that jump box can connect to MySql server. However, your local can not connect to MySql server directly, but you can connect to jump box. You want to connect to MySql server from local. In this case, we can use port forward to solve the problem.

Here is config file:

# assume your jump box has ip 55.22.33.44
# assume MySql server has ip 10.10.44.43
Host mm-jumpbox
HostName 55.22.33.44
User ec2-user
IdentityFile ~/.ssh/NOC.pem
LocalForward 3308 10.10.44.43:3306

After you put the config into .ssh, you can

#step one
ssh mm-jumpbox

#step two, in another terminal. here 3308 match 3308 in config. 
#Therefore called port forward
mysql -h 127.0.0.1 -P 3308 -u mysql-user-name --password='abdbbdbbd'

Friday, 7 May 2021

PHP composer

How to install, remove and update one library using PHP composer for an existing project

You have a existing PHP project and you have installed lots of libraries using composer. There are composer.json and composer.lock already.

use phpunit library as example

1. To install new one (--dev flag is used to put this library in require-dev section of compose.json)

composer require --dev phpunit/phpunit ^8

2. To remove old one

compose remove phpunit/phpunit

3. To update one

//without depency update
composer require --dev phpunit/phpunit ^9

We can check phpunit version

// go to vendor/bin
./phpunit --version
//with depency update
./composer.phar -W require guzzlehttp/guzzle 7.4.5

4. To install or update for all libraries

Davey Shafik has a good explanation. Composer: It’s All About the Lock File

Version in compose.json

Tilde Version Range (~) - ~1.2.3 is equivalent to >=1.2.3 <1.3.0
Caret Version Range (^) - ^1.2.3 is equivalent to >=1.2.3 <2.0.0

Search packages

Search Packages

Composer detected issues in your platform

Once I got Composer detected issues in your platform: Your Composer dependencies require a PHP version ">= 7.3.0". You are running 7.2.24-0ubuntu0.18.04.9.

some libraries have platform requirement. In the vendor composer directory, there is a file called platform_check.php. This file will be updated when you install some libraies.

use this flag --ignore-platform-reqs when do compose install to avoid update that file. Also, composer will not check php version when install. Be sure you know what you are doing.

 composer install --ignore-platform-reqs
./composer.phar require --ignore-platform-reqs -W  google/cloud ^0.257

Another option is set a flag in the compose.json. Then that file will not be generated.

"config": {
        "platform": {
            "php": "8.0.22"
        },
        "platform-check": false
    },

Clear composer cache

After a new libray is installed using composer, system seems not find the library. It may be because of cache. Can clear cache by the command below

composer clearcache

See composer tree

composer show --tree

dump-autoload

If new classess are added to classmap package, need to run the command below to update autoload. It will update vendor/composer/autoload_classmap.php

composer dump-autoload

A use case is with PHP Propel 2. In compose.json, we need to add classmap directory.

autoload": {
    "classmap": ["src/lib/Propel/generated-classes/"]}

After generated class using propel model:build, we need to run composer dump-autoload. Otherwise, may not be able to auto load these generated classes.

Propel class inherit debug for class not found

step one: om
step two: convert-conf

Lock file prevent our project from updating library

I want to update aws sdk for PHP to the latest version. Because there is a lock file, I can not remove the old version of aws sdk for PHP. If I remove the lock file first and do a composer install,I can install the latest version of aws PHP sdk. However, some functions of our project do not work. The following is how I solved this problem.

  • Find root cause is that we need "revenuewire/dynamodb-orm": "2.0.5" to make our project work
  • Currently the compose.json has this: "revenuewire/dynamodb-orm": "^2.0". If install without the lock file, it will install revenuewire/dynamodb-orm": "2.0.6" and will break the project
  • Modify compose.json. Change "revenuewire/dynamodb-orm": "^2.0" to "revenuewire/dynamodb-orm": "2.0.5". That means fix the version for revenuewire/dynamodb-orm
  • remove the compose.lock
  • compose install. Everything are working and aws PHP sdk is upgraded.

Show version of the package installed

composer show aws/aws-sdk-php

Thursday, 5 November 2020

Run Jenkins on Ubuntu

To start:

sudo service jenkins start

To stop:

sudo service jenkins stop

To restart:

sudo service jenkins restart

To check status:

sudo systemctl status jenkins 

To access: localhost:8080

use Pipe line to build git branch

  1. create a multiple scan pipe line item in Jenkins (branch sources in local format: file:///home/leo2019/play_jenkins)
  2. global tool config to add git excutable
  3. create a Jenkinsfile in the git repository

Find plugins installed

  1. go to http://localhost:8080/script
  2. past the groovy script below
  3. run the script
Jenkins.instance.pluginManager.plugins.each{
  plugin -> 
    println ("${plugin.getDisplayName()} (${plugin.getShortName()}): ${plugin.getVersion()}")
}

Install Jenkins on Docker

  1. stop jenkin if other jenkin is running
  2. Go to home directory
    mkdir jenkins_home
  3. //let folder own by jenkin. If log into container
     //and issue id command, got:  
     //uid=1000(jenkins) gid=1000(jenkins) groups=1000(jenkins)
      cd jenkins_home
      sudo chown -R 1000:1000 .
  4. docker run -d -p 49001:8080 -v ~/jenkins_home:/var/jenkins_home --name jenkins jenkins/jenkins
  5. http://localhost:49001
  6. user name: admin password in: /var/lib/jenkins/secrets/initialAdminPassword

To Start container

docker container start jenkins

To Stop container

docker container stop jenkins

Thursday, 12 September 2013

Struct and pointer sample

#include <stdio.h>

typedef struct Table Table;

struct Table {
        int Height;
        int  Width;
};

Table *  create_table(int height, int width)
{
    Table *mytable_ptr;

    Table my  = {height, width};

    mytable_ptr = &my;

    return mytable_ptr;

}



int main()
{
        Table  * my;
        my = create_table(10, 20);

       int height = my->Height;

       printf("h%d\n", height);
}

Thursday, 6 June 2013

Enviorment variables

1. View all environment variables
export -p

2. View individual variables
echo $variable_name

3. Set variable dynamically
export APPLICATION_ENV='local'

4. Set variable permanently
put export statement in .profile

Wednesday, 27 March 2013

More MySql tips

dump db

mysqldump --host='127.0.0.1' --user=root --password='passwd' --column-statistics=0  db_name>dw.sql

redirect result to a csv file

select * from usermanager_users
 into outfile '/tmp/test.csv' fields terminated by ',' enclosed by '"' 
 lines terminated by '\n';

redirect result on command line

mysql -u [username ] -p[password] [db_name] -e " SELECT * FROM [tbl_name] " > file_name

Case sensitive compare

#Will match first name Mary, but not match mary
SELECT first_name FROM  `name` WHERE first_name  REGEXP BINARY 'Mary';

Show stored procedures

SHOW PROCEDURE STATUS WHERE Db = 'db_name'\G

# show details of procedure
SHOW CREATE PROCEDURE procedure_name\G

Create a mysql user

CREATE USER lshen@'10.18.276.170' IDENTIFIED   BY 'eikdkdk123scx';

select * from mysql.user where User="lshen"\G

GRANT SELECT  ON *.* TO lshen@'10.18.276.170';

FLUSH PRIVILEGES;

Query Cache

If swap between different MySQL versions, cache may cause problem. For example, they may use the different end character. Therefore, need to flush cache.

#To check whether the query cache is present in your MySQL server
mysql> SHOW VARIABLES LIKE 'have_query_cache';

#To monitor query cache performance
mysql> SHOW STATUS LIKE 'Qcache%';

#removes all query results from the query cache
mysql> RESET QUERY CACHE ;

Slove authentication method unknown to the client issue

Error message: Unable to open PDO connection [wrapped: SQLSTATE[HY000] [2054] The server requested authentication method unknown to the client.

root cause:This error is coming from MySQL 8.x’s new default auth‐plugin (caching_sha2_password) which older PHP/MySQL clients (PDO_MYSQL via mysqlnd) don’t yet understand.

source database server is using Mysql8.4. data warehouse is using php7.3

Solution: upgrade data warehouse to use PHP 8.0

<       apk add bash curl php7-mysqlnd php7-fileinfo php7-soap php7-pdo_mysql php7-xmlwriter mysql-client \
<        php7-dom php7-tokenizer msmtp gnu-libiconv ncurses util-linux && \
---
>        apk add bash curl php8-mysqlnd php8-fileinfo php8-soap php8-pdo_mysql php8-xmlwriter mysql-client mariadb-connector-c \
>        php8-dom php8-tokenizer msmtp gnu-libiconv ncurses util-linux && \

Friday, 14 December 2012

PHP cheating sheet

Cope File to Remote Server Using PHP

1. install php ssh2 extension

in ubuntu

apt-get install libssh2-1-dev libssh2-php

 test if php ssh2 installed

file_exists('ssh2_connect')

2. find finger print of remote server
 connect to the server
 ssh-keygen -lf /etc/ssh/ssh_host_rsa_key.pub
the second part is fingerprint

3. using ssh2 to scp files

$conn = ssh2_connect('remote-server-name', 22);

$fingerprint = ssh2_fingerprint($connection,
SSH2_FINGERPRINT_MD5 | SSH2_FINGERPRINT_HEX);

if ($fingerprint != $knowprint) {
    die("middle man found");
}
ssh2_auth_password($conn, 'username', 'password');

ssh2_scp_send($conn, 'file_name_in_local, 'file_name_in_remote', 0777);

Use call_user_func function

class Hello {
    public static function say_hello()
    {
        echo "Hello World!\n";
    }
}

//There are three ways to use call_user_func

//seperate class name and function
call_user_func(array("Hello", 'say_hello'));

//combine class name and function into one string
call_user_func('Hello::say_hello');

//create a object
call_user_func(array(new Hello(), 'say_hello'));

PHP extension

what extensions have been loaded

php -m 

In Alpine, extension is here /usr/lib/php/modules

Thursday, 6 December 2012

Linux Commands

Limit resource a user can access

edit file /etc/security/limits.conf

//show limits
ulimit -a

See differences between two folders

diff -rq folder1 folder2

Find file in mac or Linux

find ./ -name platform_check.php

Open image

open test.png
open test.jpg

curl

//curl to redirect content to file
curl  --output leo.jpg https://checkout.whahaha-content.com/images/unionpay.png

created nested directories

The -p flag will create nested directories if directories do not exist. If exist, do nothing.

//mkdir: leo: No such file or directory
mkdir leo/test

//this works
mkdir -p leo/test

sed command

replace all occurences of a pattern

//ie flag will replace in place and that file will be modified
sed  -ie 's/unix/linux/g' my.txt

//all leo in my.txt will be replaced by developer. Also will make a backup file called my.txt.bak
sed -i.bak "s/leo/developer/g" my.txt

find which process is using a port number in mac

lsof -nP -iTCP -sTCP:LISTEN | grep <port-number>

In mac count items one level below current directory

find . -mindepth 1 -maxdepth 1 | wc -l

Saturday, 1 September 2012

Configure dual monitor

In Ubuntu 12.04.1 LTS

Click Dash home, and type "display". Launch the display configure program.

In Window 7

Press window command key+p to launch the display configure program.

PhpStorm

Find usage for a function in the project

Select a symbol for which you want to find usages, right-click the symbol, and select Find Usages from its context menu.

Find functions in a file
View | Tool Windows | Structure
Variable type hint
/** @var RouteCollectorProxy $app */
    public static $app;

Tuesday, 14 August 2012

SVN Tip

SVN Patch

Run command in project root

1. Create a patch
svn diff > mypatch.diff

2. apply a patch
download patch

patch -p0 -i fix_path.diff

3. reverse patch
 patch -p0 -R -i fix_path.diff

4.  How to revert svn commit:
The following two commands will do the trick

svn merge -r 101:100 http://svn.example.com/repos/calc/trunk
svn ci



5. SVN Blame

svn blame svn://repositoy/filename
                       

Thursday, 2 August 2012

Set up localhost server

  1. npm init
  2. npm install express
  3. create a index.html and index.js
  4. node index.js
  5. http://localhost:5000
//index.js
const express = require('express');
const app = express();
const port = 5000;

app.get('/', (req, res) => {
    res.sendFile('index.html', {root: __dirname});
});

app.listen(port, () => {
    console.log(`Now listening on port ${port}`);
});

Create a post api using express

const express = require('express');
const app = express();

//middleware to parse incoming request with json payload
app.use(express.json());

//middleware to parse incoming request with url encoded payload
app.use(express.urlencoded({ extended: true }));

const port = 5000;

app.get('/', (req, res) => {
    res.sendFile('index.html', { root: __dirname });
});

app.listen(port, () => {
    console.log(`Now listening on port ${port}`);
});

app.post('/api/create_transaction', (req, res) => {
    const body = req.body;
    console.log("body:", body);
    res.status(201).json([]);
});

Sunday, 29 July 2012

Change wireless password

1. connect a computer with router by cable
2. go to http://192.168.1.254
3. log in as admin
4. go to wireless ->security
5. change password