What are Tokens in programming/keywords/identifiers/operators/saperators/literals

What are Tokens in programming

Smallest individual element of a program is called as Token. Everything you see inside a program is a token.
For example - Suppose an English sentence. "C language is an awesome language. C was developed by Dennis Ritchie at AT&T Bell labs in 1972."
The above sentence is made of Alphabets (a-z A-Z), Blank spaces, Digits (0-9) and special characters (full stop in our case). These are building blocks or basic elements of our sentence. Similarly there are various basic programming elements that makes any program.
There are five types of tokens.
  1. Keywords
  2. Identifiers
  3. Operators
  4. Saperators
  5. Literals

  6. What are Keywords?

    Keyword is a reserved word whose meaning is already defined by the programming language. We cannot use keyword for any other purpose inside programming. Every programming language have some set of keywords.
    Examples: int, do, while, void, return etc(Note: These keywords are common to C and C influenced languages).

  7. What are Identifiers?

    Identifiers are the name given to different programming elements. Either name given to a variable or a function or any other programming element, all follow some basic naming conventions listed below:
    1. Keywords must not be used as an identifier.
    2. Identifier must begin with an alphabet a-z A-Z or an underscore_ symbol.
    3. Identifier can contains alphabets a-z A-Z, digits 0-9 and underscore _ symbol.
    4. Identifier must not contain any special character (e.g. !@$*.'[] etc.) except underscore _.

    Examples of some valid identifiers

    num, Num, _num, _Num, num1, Num1, _num1, _Num1, _1num, _1Num, _num_, number_to_add
    etc.

    Examples of some invalid identifiers

    1num, number to add, 1_num, num-to-add, num@
    etc.

    What are Operator?

    Operators are the symbol given to any arithmetical or logical operations. Various programming languages provides various sets of operators some common operators are:
    Lets suppose two variables a=10, b=5
    - operator subtracts second operand from first i.e. a - b and results 5.
    OperatorDescriptionExample
    Arithmetic operator
    Arithmetic operator are used to perform basic arithmetic operations.
    +Adds two operand.a + b gives 15
    *Multiplies two operands.a * b gives 50
    /Divides two operands.a / b gives 2
    %Modulus operator divides the first operand from second and returns the remainder. It is generally used for checking divisibility.a % b gives 0 (As 10/5 will have 0 remainder)
    Assignment operator
    Assignment operator is used to assign value to a variable. The value is assigned from right to left.
    =Assigns value from right operand to left operand.a = 10 will assign 10 in a

    Relational operator

    Relational operator are used to check relation between any two operands. Whether any of them is greater, equal or not equal.
    >If value of left operand is greater than right, returns true else returns false(a > b) will return true
    <If value of right operand is greater than left, returns true else returns false(a < b) will return false
    ==If both operands are equal returns true else false(a == b) will return false
    !=If both operands are not equal returns true else false.(a != b) will return true
    >=If value of left operand is greater or equal to right operand, returns true else false(a >= b) will return true
    <=If value of right operand is greater or equal to left operand, returns true else false(a <= b) will return false
    Logical operator
    Logical operator are used to combine two boolean expression together and results in a single boolean value according to the operand and operator used.
    &&Used to combine two expressions. If both operands are true or Non-Zero, returns true else false((a>=1) && (a<=10)) will return true since (a>=1) is true and also (a<=10) is true.
    ||If any of the operand is true or Non-zero, returns true else false((a>1) || (a<5)) will return true. As (a>1) is true. Since first operand is true hence there is no need to check for second operand.
    !Logical NOT operator is a unary operator. Returns the complement of the boolean value.!(a>1) will return false. Since (a>1) is true hence its complement is false.
    Bitwise operator
    Bitwise operator performs operations on Bits(Binary level). Lets suppose a = 10, b = 5
    a = 0000 1010 (8-bit binary representation of 10)
    b = 0000 0101 (8-bit binary representation of 5)
    &Bitwise AND performs anding operation on two binary bits value. If both the values are 1 then will result is 1 else will result in 0.
      0000 1010
& 0000 0101
____________
  0000 0000
    |Bitwise OR returns 1 if any of the two binary bits are 1 else returns 0.
      0000 1010
| 0000 0101
___________
  0000 1111
    ^Bitwise XOR returns 1 if both the binary bits are different else returns 0.
      0000 1010
^ 0000 0101
___________
  0000 1111
    ~Bitwise COMPLEMENT is a unary operator.It returns the complement of the binary value i.e. if the binary bit is 0 returns 1 else returns 0.
    ~ 0000 1010
___________
  1111 0101
    <<Bitwise LEFT SHIFT operator is also unary operator. It shift the binary bits to the left. It inserts a 0 bit value to the extreme right of the binary value. Or we may say it generally multiplies the value with 2.
      0000 1010 << 2 
= 0010 1000
    >>Bitwise RIGHT SHIFT operator is an unary operator. It shifts the binary bits to the right. It inserts a 0 bit value to the extreme left of the binary value. Or we may say it generally divides the value with 2.
      0000 1010 << 2
= 0000 0010
    Increment/Decrement operator
    Increment/Decrement operator is a unary operator used to increase an integer value by 1 or decrease it by 1. Increment/decrement operator are of two types Postfix and Prefix.
    ++Increment operator will add 1 to an integer value.a++ will give 11
    ++a will also give 11
    --Decrement operator will subtract 1 from an integer value.a-- will give 9
    --a will also give 9
    Conditional/Ternary operator
    Ternary operator as a conditional operator and is similar to simple if-else. It takes three operand.
    ?:It is used as conditional operator. Syntax of using ternary operator:
    (condition) ? (true part) : (false part)    		b = (a>1) ? a : b;
    will store the value 10 in b as (a>1) is true hence true part will execute, assigning the value of ain b.

    What are Separators?

    Separators are used to separate different programming elements. The various types of separators used in programming are:
     (Space) \t(Tab) \n(New line) . , ; () {} []

    What are Literals?

    Literals are constant values that are used for performing various operations and calculations. There are basically three types of literals:

    1. Integer literal

      An integer literal represents integer or numeric values.
      Example: 1, 100, -12312 etc

    2. Floating point literal

      Floating point literal represents fractional values.
      Example: 2.123, 1.02, -2.33, 13e54, -23.3 etc

    3. Character literal

      Character literal represent character values. Single character are enclosed in a single quote(' ') while sequence of character are enclosed in double quotes(" ")
      Example: 'a', 'n', "Hello", "Hello123" etc.

Comments

Post a Comment

Popular Posts