Cloud Textract - Extract Text and Data as SaaS Documentation

Version 1.0.1 Last Documentation Update - 10/02/2022

Welcome

Thank you for purchasing Cloud Textract - Extract Text and Data as SaaS Laravel Script. We covered almost everything in this document that how easily you can setup this script. We really do appreciate every sale. If you like our work please do not forget to rate it. It helps us in developing new and better cloud solutions.

Author: Berkine

Demo: Cloud Textract - Extract Text and Data as SaaS Demo

Video Demo

Watch video demostration of how to use the script and learn about its features

Requirements

Before installation, your server must have following requirements to run the script properly

PHP v7.4
PHP Mbstring Extension
PHP FileInfo Extension
PHP PDO Extension
PHP JSON Extension
PHP CURL Extension
Support for MySQL v5.7 and Up

Installation Instructions

This is a general installation overview of the script, there is a deep dive section provided for each part in the left menu bar.

Follow the steps as explained, and you should be all set. If you face any challenges with installtion steps, reach out to us for a support.

Setup overview:

Step 1: Unzip and upload only the contencts of cloudtextract to the root directory of your hosting server- Ex: /home/username/public_html/. Which will make it reachable as follows: http://yourwebsitename/
Step 2: Make sure your domainname points to the public folder only. So in your settings, path for your domain must be: htts://yourwebsitename/public. In case if you don't point your domain to public folder, we also include custom .htaccess file in the root directory for your convenience, it will automatically point requests to public directory in the script.
Step 2a: Warning! Some hosting platform hide files starting with '.' (hidden files), so make sure that in your settings of your cPanel you enable this option.
Step 2b: Warning! If your local Windows/Linux/MaxOS environment does not show hidden files, and you are planning to upload the script via FTP, make sure you enable viewing hidden files locally first even before unzipping, otherwise you will not upload all hidden files such as .env/.htaccess to your FTS server simply because your local environment will not see them.
Step 3: Everything inside of the public folder is accessible by anyone, so make sure you don't put anything sensitive there such as google json keys, etc.
Step 4: After uploading all files and making sure that domain name has proper path set, create Mysql Database and proper user in case if you don't already have one, to access this database. You can either create manually via your phpMyAdmin panel or use phpMyAdmin Wizard in your cPanel to create one.
Step 5: Now you are ready to run the install script. Go to https://yourwebsitename/install or if you skipped Step 2, go to https://yourwebsitename/public/install.
Step 6: This will start the installation process, click Start Intallation button.
Step 7: Make sure required php packages are installed and enabled on your server. Click Next to continue.
Step 8: Make sure listed folder are publicly accessbile. Click Next to continue.
Step 9: For database configuration, enter your newly created database credentials. DB Server: localhost, unless your database resides somewhere else. MySQL DB Port: 3306, unless you are using some other port.
Step 10: On the final step enter your Envato License key for this product and also your Envato Username. In case if you don't know where to get your license, you can check this article.
Step 11: After installation is complete, you are now fully ready to start. The default login name is: admin@example.com and default password is: admin12345. You can change it after loggin in in your profile settings.
Step 12: Login as admin and configure SMTP Settings under Admin Panel - General Settings - SMTP Settings. Make sure your SMTP settings work by sending a test email. Without correct SMTP settings, you cannot activate email verification feature for new users.
Step 13: To start using Cloud Textract features, activate AWS and enter required API Keys and Amazon S3 Bucket under Admin Panel - TTS Management - TTS Configuration.
Step 14: To get your results for all PDF files from AWS, setting up CRON Job correctly is very important. Warning: Every hosting has different CRON setup/configuration options. under the CRON Job Setup tab on left, we included instructions based on our hosting provider Namecheap. Our hosting doesn't allow to create CRON tasks in 1 minute internal (only 5 minutes), but if yours does allow, for user experience it is highly recommended to set CRON task running schedule to be in 1 Minute Interval. Otherwise set whichever is the minimum allowed value in your hosting.

How to Update the script to v1.0.1

With version v1.0.1 we added auto update feature, going forward update process will be very easy(one click update), meanwhile, please follow these steps if you already have v1.0.0 installed and running and don't want to do a fresh install.

How to Update:

Step 1: Important: Make sure to backup your database and .env file always before any update, and don't forget to backup your customized frontend files, also if you have custom .htaccess file in the root folder, back it up as well.
Step 2: If you have previously activated the script, copy the .lic file in the root directory to some other temporary place. Reminder: You license is strictly bounded with your domain name.
Step 3: Copy install file under storage directory to some other temporary place.
Step 4: Replace all files with the v1.0.1
Step 5: Copy paste back the .lic file to the root directory, make sure it has '.' infront of lic filename. Copy past installed file to the storage directory.
Step 6: Replace .env in the root direcotry with the .env file that you backed up before and include this line into your .env file and click save:
```
CONFIG_FILE_PREVIEW=disable
```
Step 7: Login with your admin credentials, and type: https://domainname/update/now. This will migrate all newly create tables.
Step 8: (Optional) In your .env file, change APP_VERSION to be equal to v1.0.1
Step 9: Update Complete.

Database Setup in phpMyAdmin

To run admin panel properly and fetch the application useage data and to set the settings you will need to setup MySQL database 5.7.

How to create new Database in phpMyAdmin:

Step 1: Login into phpMyAdmin.
Step 2: Click on New button to create a new database.
Step 3: Provide database name and select utf8_general_ci as collation and click create button.
Step 4: Make sure you have database user who can access this newly created database, if not you will need create one with all permissions and set a password
Step 5: Also, to avoid any issues with database setup, make sure your newly created database has InnoDB set as storage type
Step 6: Return to the Install process and enter your newly created database credentials. If using local server, you can set database host to localhost, and for mysql port use 3306

AWS Account Creation

Getting started with AWS. If you already have AWS Account, skip to next section.

How to Create AWS Account:

Step 1: Open the Amazon Web Services home page.
Step 2: Choose Create an AWS Account.
Note: If you signed in to AWS recently, choose Sign in to the Console. If Create a new AWS account isn't visible, first choose Sign in to a different account, and then choose Create a new AWS account.
Step 3: Enter your account information, and then choose Continue. Be sure that you enter your account information correctly, especially your email address. If you enter your email address incorrectly, you can't access your account.
Step 4: Choose Personal or Professional.
Note: Personal accounts and professional accounts have the same features and functions.
Step 5: Enter your company or personal information.
Important: For professional AWS accounts, it's a best practice to enter the company phone number rather than a personal cell phone. Configuring a root account with an individual email address or a personal phone number can make your account insecure.
Step 6: Read and accept the AWS Customer Agreement.
Note: Be sure that you read and understand the terms of the AWS Customer Agreement.
Step 7: Choose Create Account and Continue.
Step 8: On the Payment Information page, enter the information about your payment method, and then choose Verify and Add.
Note: If you want to use a different billing address for your AWS billing information, choose Use a new address before you choose Verify and Add
Important: You can't proceed with the sign-up process until you add a valid payment method.
Step 9: Verify your phone number. Enter a phone number where you can be reached in the next few minutes.
Step 10: In a few moments, an automated system contacts you.
Step 11: Enter the PIN you receive, and then choose Continue.
Step 12: On the Select a Support Plan page, choose one of the available Support plans. For a description of the available Support plans and their benefits, see Compare AWS Support plans.
Step 13: After you choose a Support plan, a confirmation page indicates that your account is being activated. Accounts are usually activated within a few minutes, but the process might take up to 24 hours.
Step 14: You can sign in to your AWS account during this time. The AWS home page might display a Complete Sign Up button during this time, even if you've completed all the steps in the sign-up process.
Step 14: When your account is fully activated, you receive a confirmation email. Check your email and spam folder for the confirmation email. After you receive this email, you have full access to all AWS services.

AWS IAM User Creation

You will need IAM user with programmatic access and with Transcribe and also S3 Read & Write Policies attached.

Creating AWS IAM Users (via AWS Console):

Step 1: Sign in to the AWS Management Console and open the IAM console at https://console.aws.amazon.com/iam/.
Step 2: In the navigation pane, choose Users and then choose Add user.
Step 3: Type the user name for the new user. This is the sign-in name for AWS.
Note: User names can be a combination of up to 64 letters, digits, and these characters: plus (+), equal (=), comma (,), period (.), at sign (@), and hyphen (-). Names must be unique within an account.
Step 4: Select the type of access this set of users will have. Select programmatic access.
Step 5: Choose Next: Permissions.
Step 6: On the Set permissions page, specify how you want to assign permissions to this set of new users. Choose Attach existing policies to user directly
Step 7: Seach for AmazonS3FullAccess and AmazonTextractFullAccess, select both afterwards select click Next: Review
Note: If you are familar with IAM Policies, you can create custom policy that will match your needs and security requirements.
Step 8: To view the users' access keys (access key IDs and secret access keys), choose Show next to each password and access key that you want to see. To save the access keys, choose Download .csv and then save the file to a safe location.
Important: This is your only opportunity to view or download the secret access keys, and you must provide this information to your users before they can use the AWS API. Save the user's new access key ID and secret access key in a safe and secure place. You will not have access to the secret keys again after this step.
Step 9: Include your Access Key and Secret Access Key to the AWS Credentials Settings tab in the admin panel as shown below.

Under Admin -> TTS Management -> TTS Configuration -> AWS, include your Access and Secret Access Keys:

Amazon S3 Bucket Creation

You will need an S3 Bucket with public read & write access in the region that you prefer.

A bucket is owned by the AWS account that created it. By default, you can create up to 100 buckets in each of your AWS accounts. If you need additional buckets, you can increase your account bucket limit to a maximum of 1,000 buckets by submitting a service limit increase. For information about how to increase your bucket limit, see AWS Service Limits

Create an S3 bucket:

Step 1: Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/.
Step 2: Choose Create bucket.
Step 3: Enter Bucket Name and Select AWS Region where you would like to store your files and click Next. Note: Bucket name must be unique at a global level.
Step 4: In the Set Permissions step, you need to turn off all 4 options if you want to make your S3 bucket completely public.
Step 5: Review rest of the settings, or you can leave them as they are and click Create Bucket.
Step 6: Include your Bucket Name and Select your AWS Region in the AWS Credentials Settings tab in the admin panel as shown below.

Under Admin -> TTS Management -> TTS Configuration -> AWS, include your S3 Bucket Name and select your AWS Region where bucket is hosted:

Amazon S3 Bucket Configuration

After successfully creating a bucket, add these addtional features for your S3 bucket.

Check these 2 steps:

Step 1: Make sure your Permissions -> Block Public Access looks at least like this and has all 4 options turned off.

Step 2: Add CORS settings as shown below in the Permissions -> Cross-origin resource sharing (CORS) Configuration section and click Save.

Copy paste these CORS settings to your bucket:

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "POST",
            "PUT",
            "GET",
            "HEAD",
            "DELETE"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": [
            "ETag"
        ],
        "MaxAgeSeconds": 30000
    }
]

Step 3: Go to your newly created S3 Bucket click on Permissions -> Object Ownership and click Edit.
Step 4: Select ACLs enabled and Object writer and click Save Changes.

Paypal Setup

For accepting payments both for Monthly Subscriptions and for Prepaid options you can use Paypal.

You will need to include your stripe credentials under Admin -> Finance Management -> Payment Settings -> Paypal. Include all your Paypal Credentials, and don't forget to active the Paypal checkbox(es):

PayPal Client ID
PayPal Client Secret
Domain Name for Paypal Webhook URI
Paypal Webhook ID

Include required data and click Save button:

How to get Paypal Client ID and Paypal Client Secret:

Login into your dashboard at developer.paypal.com
On the left menu bar, select Dashboard -> My Apps & Credentials -> Create New App
Provide desired app name and click create
Paypal will provide you Client ID and Client Secret for your newly created app as shown in the demo image below, in your Polly script store them under Management -> Payment Settings -> Paypal Client ID and Client Secret fields.

Create Paypal Webhook with select events:

On the same page as expained above, under Dashboard -> My Apps & Credentials, select your newly created app, and scroll to the bottom of the page and you reach "Sandbox Webhooks", click "Add Webhook"
Include your domain name followed by /webhooks/paypal
Ex: https://www.domainname.com/webhooks/paypal
Important! Make sure you have the same path included for Paypal Webhook URI under Finance Management -> Payment Settings -> Paypal
Select following webhook events for subscriptions and click save:
- BILLING.SUBSCRIPTION.CANCELLED
- BILLING.SUBSCRIPTION.SUSPENDED
- BILLING.SUBSCRIPTION.PAYMENT.FAILED
- BILLING.SUBSCRIPTION.EXPIRED
Save the Webhook ID under Finance Management -> Payment Settings -> Paypal -> Paypal Webhook ID

Stripe Setup

For accepting payments both for Monthly Subscriptions and for Prepaid options you can also use Stripe.

You will need to include your stripe credentials under Admin -> Finance Management -> Payment Settings -> Stripe. Include all your Stripe Credentials, and don't forget to active the Stripe checkbox(es):

Stripe Key
Stripe Secret Key
Domain Name for Stripe Webhook URI
Stripe Webhook Secret

Include required data and click Save button:

How to get Stripe API Key and Stripe Secret Key:

Login into your dashboard at dashboard.stripe.com
On the left menu bar, select Developers -> API Keys -> You will see your Publishable Key (API Key) and Secret Key
Copy and paste them to Cloud Polly, under Finance Management -> Payment Settings -> Stripe

Create Stripe Webhook with select events:

Login into your dashboard at dashboard.stripe.com
On the left menu bar, select Developers -> Webhooks -> Add Endpoint
Create a Webhook in your Stripe Developer Account
Include your domain name followed by /webhooks/stripe
Ex: https://www.domainname.com/webhooks/stripe
And add following two events:
1. invoice.payment_failed
2. customer.subscription.deleted
After your webhook is created, copy Webhook URL and Webhook Secret to Admin -> Finance Management -> Payment Settings -> Stripe to the respective fields.

Promocodes Setup

(Optional) You can also create promocodes that you can share with your customers

NOTE: Promoces can be used only with Prepaid Plans (credit Top up feature)

Create Promocode:

Under Finance Management -> Promocodes -> Click on "CREATE NEW PROMOCODE" button.

Enter unique promocode name
Select "Percentage Discount" or "Fixed Discount" options
Enter discount amount. Warning! Do not use % sign for "Percentage Discount", use only integer values
Enter quantity of promocodes that you want to release
Select expiration date for these promodes

Example of a filled promocode:

NOTE: 100% Percentage Discount will not redirect to any payment gateways, and instead will automatically assign respective credits to users account. If with 100% Percentage Discount, Stripe is selected, user still needs to enter credit card informaiton, but nothing will be processed and no money will be deducted from users account bank account.

Prepaid Plan Setup

You can create as many prepaid plans as you want.

All prepaid plans are one time payments, can be paid either by Paypal or Stripe depending on which one you activated.

How to create Prepaid Plan:

Step 1: Go to "Admin Panel" -> "Finance Management" -> "Prepaid Plans" -> click on "Create New Pre Paid Plan" button.
Step 2: Fill up the all required fields.
Step 3: Note: Plan Name must be a unique string value, or digits. Ex: '100,000'
Step 4: After filling up the fields, click "Create" button. Example: your newly created prepaid will looks like this:

Subscription Plans Setup

You can create as many subscription plans as you want.

Currently only monthly payment subscription options are available, and due to the "pay as you go" model of the all Cloud Service Providers, we recommend to avoid having yearly subscription, unless it is capped to some limited number of characters per month.

How to create Subscription Plan:

Step 1: Go to your Paypal and/or Stripe account and Create New Subscriptions/Plans.
Step 2: Copy those Subscription/Plan IDs.
Step 3: Go to "Admin Panel" -> "Finance Management" -> "Subscription Plans" -> click on "Create New Subscription" button.
Step 4: Fill up the all required fields and inlcude Paypal/Stripe - Subscription/Plan IDs.
Step 5: Note: Plan Name must be a unique string value
Step 6: Plan descriptions must be a comma separated string values
Step 7: After filling up the fields, click "Create" button. Example: your newly created subscription will looks like this:

Create Paypal Plan ID

For Monthly Subscription Plans you will need to include Paypal Plan IDs if you want to receive payment by Paypal.

If you know how to create and get Paypal Plan ID in your Paypal Merchant Account, you can skip this instructions.

How to create Paypal Plan ID for a Subscription Plan:

Step 1: Go to your Paypal and/or Stripe account and Create New Subscriptions/Plans.
Step 2: Copy those Subscription/Plan IDs.
Step 3: Go to "Admin Panel" -> "Finance Management" -> "Subscription Plans" -> click on "Create New Subscription" button.
Step 4: Fill up the all required fields and inlcude Paypal/Stripe - Subscription/Plan IDs.
Step 5: Note: Plan Name must be a unique string value
Step 6: Plan descriptions must be a comma separated string values
Step 7: After filling up the fields, click "Create" button. Example: your newly created subscription will looks like this:

Create Stripe Product ID

For Monthly Subscription Plans you will need to include Stripe Product IDs if you want to receive payment by Stripe.

If you know how to create and get Stripe Product ID in your Stripe Account, you can skip this instructions.

How to create Stripe Product ID for a Subscription Plan:

Step 1: Go to your Paypal and/or Stripe account and Create New Subscriptions/Plans.
Step 2: Copy those Subscription/Plan IDs.
Step 3: Go to "Admin Panel" -> "Finance Management" -> "Subscription Plans" -> click on "Create New Subscription" button.
Step 4: Fill up the all required fields and inlcude Paypal/Stripe - Subscription/Plan IDs.
Step 5: Note: Plan Name must be a unique string value
Step 6: Plan descriptions must be a comma separated string values
Step 7: After filling up the fields, click "Create" button. Example: your newly created subscription will looks like this:

SMTP Configuration

For sending all emails, such as email verification, support ticket creation etc, SMTP needs to be configured.

All the SMTP settings can be configured and set in Admin Panel under General Settings -> SMTP Settings.

Required SMTP Settings:

SMTP Host
SMTP Port
SMTP Encryption
SMTP EMail Login (Username)
SMTP EMail Password
SMTP Sender Email Address
SMTP Sender Name

Include Required SMTP Settings and click Save Changes button:

You can also test your SMTP Settings immediately via Test button:

If you are planning to use Gmail for sending emails:

In your Gmail Account settings, under Security tab, make sure your Less secure app access feature is turned on.

Google Analytics Configuration

Analyze in-depth detail about the visitors on your main frontend page.

Setup Google Analytics:

Step 1: Visit Google Analytics page.
Step 2: On the left menu select Admin.
Step 3: Click Create Property, **select Web and then Continue.**.
Step 4: After that it will appear screen with information for your website.
Step 5: Fill it and then click Create.
Step 6: Include Tracking ID into Google Analytics Tracking ID field in Admin Panel under General Settings -> Global Settings -> Google Analytics, and don't forget to enable the Google Analytics checkbox and click Save button.

Google Maps Configuration

To use the Maps JavaScript you must have an API key. The API key is a unique identifier that is used to authenticate requests associated with your project for usage and billing purposes.

To Get a Maps API Key:

Step 1: Go to the Google API Console
Step 2: Click the project drop-down and select or create project for which you want to add an API key and give it a project name.
Step 3: If you are using Google API for the first time you will see this screen after creating the project:
Step 4: Click Enable Apis and Services button or select Library button on the left.
Step 5: Find Maps JavaScript API and enable it as shown below.
Step 6: Click the menu button and select Credentials and click Create credentials > API key.
Step 7: The API key created dialog displays your newly created API key. Copy and paste it as shown in Step 8. The new API key is listed on the Credentials page under API keys.

Important: Remember to restrict the API key before using it in production.
Step 8: Include the API key in Admin Panel under General Settings -> Global Settings -> Google Maps, and don't forget to enable the Google Maps checkbox and click Save button.

Change application restrictions:

Depending on your server/domain configuration the script maybe won't accept the HTTP referrers restrictions so try with changing the application restrictions back to None

Enable Billing on your project:

Sometimes after adding the key these features maybe won't work again. Then depending on the google account associated with the project created and maybe you will need to enable Billing.

Learn more here about it: Billing or here Getting started with Google Maps Platform

Google reCaptcha v3 Configuration

To avoid spams, all contact and registration forms have Google reCaptcha v3 enabled. It is optional feature, but strongly recommended.

To use this feature, you will need to have reCaptcha Site Key and reCaptcha Secret Key.

To Get a Google reCaptcha v3 Keys:

Step 1: Go to the Google reCaptcha Console
Step 2: Enter Label to identify your reCaptcha keys.
Step 3: For reCaptcha Type, select "reCAPTCHA v3":
Step 4: Enter your domain name.
Step 5: Accept the reCAPTCHA Terms of Service
Step 6: Click the "Submit" button.
Step 7: Copy both "Site Key" and "Secret Key".
Step 8: Include the API keys in Admin Panel under General Settings -> Global Settings -> Google reCaptcha, and don't forget to enable the Google reCaptcha checkbox and click Save button.

How to Customize Frontend

All frontend pagename.blade.php files consist of standard HTML + CSS Bootstrap structure. So it is easy to edit.

Warning: If you are planning to add heavy css customization, it would be better to create a separate css file and keep intact upon each update, otherwise next update might rewrite all previously made changes.

How to customize frontend:

Step 1: All files that are used as frontend pages or for frontend can be found under resources/views and couple files under resources/views/layouts directories as shown in the picture.
Step 2: For frontend styling we used SaSS, they can be found under resources/sass/frontend directory. If you plan to use sass files for customization don't forget to compile your sass files first to apply the effects. Otherwise, you can add custom css files into guest.blade.php below line 23. (With version 1.0.2 you will be able add directly via admin panel)
Step 3: All pages are stored direcly under resources/views directory and named accordingly. You can edit the text in any of these files as you prefer.
Step 4: If you want to add new page, make sure it has the same @extends and @section parts as in other pages.

Translate to other Languages

Cloud Polly by default is written in English Language, we also included as an example support for Spanish and French languages, translated by Google Translate (Warning! Quality of these translations might not be the best).

You can translate or add any language as you prefer.

How to Translate:

Step 1: Go to "Resources" -> "Lang" folder.
Step 2: Copy "en.json" to your desired language code. Ex: "ru.json"
Step 3: Translate only the value part, and make sure they stay within double quotes with comma at the end except the last value.
Step 4: Save your file in the same "Lang" folder.
Step 5: Go to "Config" -> "local.php" file.

Step 6: Add your new language code with proper fields: 'code', 'display', 'flag'.

'ru' => [
    'code' => 'RU',
    'display' => 'Русский',
    'flag' => 'ru'
],

CRON Job Configuration

The software requires cron jobs to be set up in order receive the results for all AWS Transcribe tasks and for GCP Transcribe tasks that are larger than 1 minute in length.

IMPORTANT: Every hosting provider might have different CRON setup page and different path for PHP bin directory. Instructions below are provided based on our hosting - Namecheap, so path to PHP Bin and path to script can be different. Check with your hosting provider if you are not aware how to setup similar CRON job in your environment.

How to create CRON job in CPANEL:

Step 1: In your Cpanel find CRON Jobs option, it could be under Advanced panel as shown in the image.
Step 2: In the CRON Jobs panel add a New Cron Job. Select the minimum time that your hosting provider allows you to run a Cron Job (minimum you can get is 1 minute). Note: Not every hosting provider allows you to run Cron Jobs every minute, our hosting provider (Namecheap) for example allows to run only every 5 minute.
Step 3: Copy paste the command show in the first red box into the Command field.

/usr/local/bin/php /home/username/YOURDOMAINPATH/artisan schedule:run

Important:

Step 4: After creating a CRON Job successfully you will see it listed among your Current Cron Jobs list:
Step 5: In case if your hosting allows you to enter entire CRON job command at ones, you can enter it in the following way (Keep in mind that if your hosting allows you to run tasks every minute, you will need to change */5 to */1):

*/5 * * * * /usr/local/bin/php /home/username/YOURDOMAINPATH/artisan schedule:run

How to create CRON job in PLESK:

You might want to follow these instructions for Plesk at least on the setup part: Plesk CRON Instructions

Initial Release v1.0.1

Released on 10th February, 2022

New: Auto Update feature added

New: Upload file preview - enable/disable option added

Update: Results and Projects table performance improvements

Update: T&C and Policy page layouts updated

Update: All results view tables updated

Fix: Projects view all results fixed

Fix: Projects metrics view fixed

Fix: Incorrect JSON result parsing fixed

Initial Release v1.0.0

Released on 1st November, 2021