Purpose

There are over 150 existing podcasts need to be imported to the NEW site.

These podcasts are subpages on the current live site.
These pages have no category and featured images attached.
No custom field to store the Episode number.
The newly developed site treats these as post type with category Podcast, and has a custom field to store the Episode number for each podcast post.

Main Steps

Current live site – Use standard WP tool to export pages to an XML file.
Set up cur-cwood.innovriver.com (cur-cwood). Copy following settings from Live CW to cur-cwood.
- Settings in General
- Settings in Permalink
- Create users as in Live.
Import the XML files to cur-cwood.
- Assign post/page creators to existing or new users.
Get the details as described in below Export/Import section.
Set up dev.innovriver (DEV) that is cloned with the image of clarewood.tdowebsites.com.au (CWtdo)
- Create users as in Live.
To import podcast to CWtdo, follow the steps in section On CWtdo below.

Export/Import

On cur-cwood

On cur-cwood, develop a php program to extract the following to a file in JSON format. One file for each podcast page.
- ID
- post_author
- post_date
- post_content
- post_title
- post_excerpt
- post_status
- post_type
- post_name
- post_parent
- post_category
- attached image – the URL should be included in the post content.
Two pages are created for the php programs:
- PHP Script – using template ‘page-php-script.php’. This is to capture user input for the episode range to be handled in the exports.
- PHP Result – using template ‘page-php-result.php’. This contains all the codes to extract data from podcast pages and extract contents from the live site. The results of extraction are displayed.
- These templates are located under the child theme directory.
The post content from the exported XML files are full of the WPB builder shortcodes, which is difficult to handle. Therefore the Simple HTML DOM library is used to extract the content from the web front end.
A directory under child theme is created:
- podcast_export/lib – stored the ‘simple-html-dom’ library.
- podcast_export/exported_json/ – stores the json files for each exported podcast.

On DEV

Create directories under the child theme:
- podcast_import/json – copy podcast json files to here
- podcast_import/logs
Two pages are created for the php programs:
- PHP Import Script – using template ‘page-php-import-script.php’. This is to capture user input for the episode range to be handled in the imports.
- PHP Import Result – using template ‘page-php-import-result.php’. This contains all the codes to import podcasts using the json files.
- These templates are located under the child theme directory.
The import process will do the following:
- Add category = podcast
- Set custom field ‘episode’ = episode number
- Set up feature image
Write CSS on style.css to style the posts.

On CWtdo

Create users as in Live.
Create directories under the child theme:
- podcast_import/json – copy podcast json files to here
- podcast_import/logs
Two pages are created for the php programs:
- PHP Import Script – copy ‘page-php-import-script.php’ from DEV.
- PHP Import Result – copy ‘page-php-import-result.php’ from DEV.
- These templates are located under the child theme directory.
The import process will do the following:
- Add category = podcast
- Set custom field ‘episode’ = episode number
- Set up feature image
Copy the json files from cur-cwood (or DEV)
Copy the relevant CSS section for converted podcasts.
Open PHP Import Script page and carry out the import operation.
After conversion of old podcast is completed, delete below items:
- Page PHP Import Script
- Page PHP Import Result
- Directory podcast_import
- page-php-import-script.php
- page-php-import-result.php

To follow up:

On DEV, Detect any Images inside content-bottom and ajust URLs. -> done
On cur-cwood, extract content as Excerpt. -> done.
Paragraphs after content image are cutoff. -> done

Codes on ‘cur-cwood’

page-php-script.php

<?php
/**
 * 
 */


if ( ! defined( 'ABSPATH' ) ) {
	exit; // Exit if accessed directly.
}

get_header(); 

?>

<html>

<body>
	<div class="cu-container">

		<div class="cu-export-header">
			<h3 style="text-align:center;padding-bottom: 50px;">Export Podcasts from current Live Site</h3>
			<h4>Enter the range of episodes for export below</h4>
		</div> <!-- cu-export-header -->

		<div class="cu-export-form">
			<form action="/php-result" method="post">
				<label for="from_ep">From Episode* : </label>
				<input type="number" name="from_ep" required><br><br>

				<label for="to_ep">To Episode* : </label>
				<input type="number" name="to_ep" required><br><br>

				<input type="submit" name="Submit">
			</form>
		</div>	<!-- cu-export-form -->

	</div> <!-- cu-container -->


	<style>
		.cu-container {
			display: block;
			width: 100%;
			padding: 30px 20px;
		}

		.cu-export-header {
			width: 100%;
			margin: 30px 0px;
		}

		.cu-export-form {
			width: 100%;
			margin: 30px auto;
		}

		.cu-export-form label:first-of-type {
			padding-right: 10px;
		}

		.cu-export-form label:nth-of-type(2) {
			padding-right: 30px;
		}

	</style>

</body>

</html>

page-php-result.php

<?php

/**
 * 
 */

require_once 'podcast_export/lib/simple_html_dom.php';

define('LIVE_LINK', 'https://clarewood.com');

date_default_timezone_set("Asia/Hong_Kong");


if (isset($_POST['Submit'])) {

	$from = $_POST['from_ep'];
	$to = $_POST['to_ep'];

?>
<div class="cu-container">
	<h3>Handling Episodes From <?=$from?> To <?=$to?></h3>	
	<br>
	<div><a href="/php-script/"><h4>Return</h4></a></div>
	<br><br>

<?php

	for($i = $from; $i <= $to; $i++) {
		$link = "/podcast/episode$i";
		$pageObj = get_page_by_path($link); 

		if (is_null($pageObj)) {
			$message = date('H:i:s'). "HKT [error] ep$i: Page has not beem imported from Live site....! \n";
			ivr_log_message($message);
			$message = "---------------\n";	
			ivr_log_message($message);

			continue;
		}
		
		$dataArr = array();
		$dataArr['episode'] = $i;
		$dataArr['author'] = get_userdata($pageObj->post_author)->user_login;
		$dataArr['created'] = $pageObj->post_date;
		$dataArr['title'] = $pageObj->post_title;
		$dataArr['excerpt'] = $pageObj->post_excerpt;
		$dataArr['status'] = $pageObj->post_status;
		$dataArr['name'] = $pageObj->post_name;

		$podcast_content = get_podcast_content($i);
		$dataArr['image'] = $podcast_content['image'];
		$dataArr['content_img_linkAArr'] = $podcast_content['content_img_linkAArr'];
		$dataArr['content'] = $podcast_content['text'];


		if ($dataArr['excerpt'] == '') $dataArr['excerpt'] = $podcast_content['excerpt'];

		//print_r($dataArr['content']);


		$json = json_encode($dataArr);

		if (file_put_contents( get_stylesheet_directory(). "/podcast_export/exported_json/episode-$i.json", $json)) {
			$message = date('H:i:s'). "HKT [info] File 'episode-$i.json' created successfully.\n\n";	
			ivr_log_message($message);
		} else  {
			$message = date('H:i:s'). "HKT [error] Error creating file 'episode-$i.json'.\n\n";
			ivr_log_message($message);
		}

		$message = "---------------\n";	
		ivr_log_message($message);


	} // for loop

} // if Submit is set
?>

</div>  <!-- cu-container -->


<?php
function get_podcast_content($Num) {
	$link = LIVE_LINK. "/podcast/episode$Num";
	$html = file_get_html($link);	

	// Get full size image link of the top image
	$images = $html->find('figure img');
	$img_srcset = $images[1]->attr['srcset'];
	$img_link = ivr_get_full_size_img_link($img_srcset);

	$content_dom = $html->find('.article-content section', 1);

	// Remove the first 3 div
	$ep_num = $content_dom->find('.wpb_wrapper', 0)->find('div', 0);
	$ep_num->remove();
	$ep_title = $content_dom->find('.wpb_wrapper', 0)->find('div', 0);
	$ep_title->remove();
	$feat_img = $content_dom->find('.wpb_wrapper', 0)->find('div', 0);
	$feat_img->remove();

	$content_dom = ivr_clean_email_addrs($content_dom, $Num);


	/**
	 * Process <img> tags to get full size image links from srcset
	 * and place these links in array.
	 * 
	 * Replace src attributes with a placeholder and set this placeholder as 
	 * key for the above array.
	 * 
	 * Remove the srcset attributes.
	 * 
	 */
	$content_imgs = $content_dom->find('img');
	$content_img_linkAArr = array();

	if (!empty($content_imgs)) {
		$j = 0;
		foreach ($content_imgs as $ci) {			
			$ci_srcset = $ci->attr['srcset'];
			$ci_link = ivr_get_full_size_img_link($ci_srcset);
			$ci->attr['srcset']	= null;
			$ci->attr['src'] = "ivr-src$j";
			$content_img_linkAArr[$ci->attr['src']] = $ci_link;
			$j++;
		} // foreach

		$message = date('H:i:s'). "HKT [info] ep$Num: $j content image link(s) extracted. \n";
		ivr_log_message($message);

	}  // if not empty
	/** End process <img> tag */

	$content = "<div class='cu-imported-podc cu-imported-podc-ep$Num'>". $content_dom->outertext. "</div> <!-- cu-imported-podc -->";

	// Get paragraphs above the line "In this Episode:" as Excerpt					
	$extracted_ps = $content_dom->find('p');
	$excerpt = "";
	foreach ($extracted_ps as $ep) {
		$p = trim($ep->plaintext);
		if (strtolower($p) == 'in this episode:') break;		
		$excerpt .= $p. ' ';
	} // foreach

	return array('text'=>$content, 'image'=>$img_link, 'excerpt'=>$excerpt, 'content_img_linkAArr'=> $content_img_linkAArr);

} // get_podcast_content()



function ivr_get_full_size_img_link($srcset) {	
	$img_srcsetArr = explode(',', $srcset);
	$url_wArr = array();

	foreach ($img_srcsetArr as $url_w) {
		$tmpArr = explode(' ', trim($url_w));  // must be trimmed		
		$url_wArr[intval($tmpArr[1])] = $tmpArr[0];  // key is width, value is url
	}

	$max_w = max(array_keys($url_wArr));  // get the max width in srcset
	return $url_wArr[$max_w];

} // ivr_get_full_size_img_link


/**
 * Clean the email addresses in the dom object, then
 * returns the cleaned dom object.
 */
function ivr_clean_email_addrs($domObj, $Num) {
	$email_addrs = $domObj->find('.pep-email');
	foreach ($email_addrs as $ea) {
		$address = $ea->innertext;
		if (strpos($address, 'with the @ sign')) {
			$address = str_replace('(Replace this parenthesis with the @ sign)', '@', $address);
		}
		$mailto ="<a href='mailto:$address' target='_blank'>$address</a>";
		$ea->outertext = $mailto;

		$message = date('H:i:s'). "HKT [fixed] ep$Num: Modified email $address in content. \n";
		ivr_log_message($message);
	} // end forearch

	return $domObj;
} // ivr_clean_email_addrs


function ivr_log_message($msg) {
	echo "<p style='color:#1b742e'>". $msg. "</p>";
	$fName = date('Y-m-d'). "_HKT-export.log";
	file_put_contents( get_stylesheet_directory(). "/podcast_export/logs/$fName", $msg, FILE_APPEND);

} // ivr_log_message


?>

<style>
	.cu-container {
		width: 80%;
		max-width: 900px;
		margin: 30px auto;
	}
</style>

Codes on ‘DEV’ and ‘CWtdo’

page-php-import-script.php

<?php
/**
 * 
 */


if ( ! defined( 'ABSPATH' ) ) {
	exit; // Exit if accessed directly.
}

get_header(); 

?>

<html>

<body>
	<div class="cu-container">

		<div class="cu-export-header">
			<h4 style="text-align:center;padding-bottom: 50px;">Import Podcasts from current Live Site</h4>
			<p>Details are given in <a href="https://innovriver.com/fai-room/import-existing-podcasts-to-new-site-new-approach/" target="_blank">HERE.</a></p>
			<h5>Enter the range of episodes for import below</h5>
		</div> <!-- cu-export-header -->

		<div class="cu-export-form">
			<form action="/php-import-result" method="post">
				<label for="from_ep">From Episode* : </label>
				<input type="number" name="from_ep" required><br><br>

				<label for="to_ep">To Episode* : </label>
				<input type="number" name="to_ep" required><br><br>

				<input type="submit" name="Submit">
			</form>
		</div>	<!-- cu-export-form -->

	</div> <!-- cu-container -->


	<style>
		.cu-container {
			display: block;
			width: 90%;
			padding: 30px 20px;
			max-width: 900px;
			margin: 30px auto;

		}

		.cu-export-header {
			width: 100%;
			margin: 30px 0px;
		}

		.cu-export-form {
			width: 100%;
			margin: 30px auto;
		}

		.cu-export-form label:first-of-type {
			padding-right: 10px;
		}

		.cu-export-form label:nth-of-type(2) {
			padding-right: 30px;
		}



	</style>

</body>

</html>

page-php-import-result.php

<?php

/**
 * 
 */


if ( ! defined( 'ABSPATH' ) ) {
	exit; // Exit if accessed directly.
}

get_header(); 

date_default_timezone_set("Asia/Hong_Kong");

require_once ABSPATH . 'wp-admin/includes/image.php';

?>
<div class="cu-container">


<?php

if (isset($_POST['Submit'])) {

	$from = $_POST['from_ep'];
	$to = $_POST['to_ep'];

?>
	<h5>Handling Episodes From <?=$from?> To <?=$to?></h5>	
	<br>
	<div><a href="/php-import-script/"><h6>Return</h6></a></div>
	<br><br>

<?php

	for($i = $from; $i <= $to; $i++) {

		//check existence
		$post = get_posts(array(
			'posts_pre_page' => 1,
			'meta_key' => 'episode',
			'meta_value' => "$i",
			'fields' => 'ids',
		));


		if (count($post)) {
			$message = date('H:i:s'). "HK [error] ep$i: Episode already exists with Post ID ". $post[0]. ". Insert skipped. \n";
			ivr_log_message($message);

		} else {
			$podc_json = file_get_contents(get_stylesheet_directory(). "/podcast_import/json/episode-$i.json");
		
			if ($podc_json) {
				$podcArr = json_decode($podc_json, true);

				$message = date('H:i:s'). "HKT [info] ep$i: Read json file OK \n";
				ivr_log_message($message);

				// Process <img> tags
				// replace placeholders with modified image urls.

				$content_img_linkAArr = $podcArr['content_img_linkAArr'];
				$mod_content = $podcArr['content'];

				if (!empty($content_img_linkAArr)) {
					foreach ($content_img_linkAArr as $key => $link) {
						$mod_link = explode('uploads', $link)[1];
						$mod_link = get_site_url(). '/wp-content/uploads'. $mod_link;
						$mod_content = str_replace($key, $mod_link, $mod_content);

						$message = date('H:i:s'). "HKT [info] ep$i: $key changed to $mod_link\n";
						ivr_log_message($message);

					} // foreach

				} // if ! empty

				// End:  Process <img> tags

				$cat_id = get_category_by_slug('podcast')->term_id;

				$author_id = get_user_by('login', $podcArr['author'])->ID;

				$args = array(
					'post_author' => $author_id,
					'post_date' => $podcArr['created'],
					'post_title' => $podcArr['title'],
					'post_excerpt' => $podcArr['excerpt'],
					'post_status' => $podcArr['status'],
					'post_name' => $podcArr['name'],
					'post_type' => 'post',
					'post_category' => array($cat_id),
					'meta_input' => array(
						'episode' => $podcArr['episode'],
						),
					'post_content' => $mod_content,	
				);

				$insert_id = wp_insert_post($args, true);

				if (!is_wp_error($insert_id)) {

					$message = date('H:i:s'). "HKT [info] ep$i: Podcast inserted to DB ($insert_id) \n";
					ivr_log_message($message);

					// Process featured image
					$fea_img_old_Url = $podcArr['image'];
					$fea_attachmt_id = ivr_attach_img_toPost($fea_img_old_Url, $insert_id, $i);

					// assign featured image to post
					if ($fea_attachmt_id != 0) {
						$thumbnail = set_post_thumbnail($insert_id, $fea_attachmt_id);
					}

					if ($thumbnail) {
						$message = date('H:i:s'). "HKT [info] ep$i: Set thumbnail is successful \n\n";
					} else {
						$message = date('H:i:s'). "HKT [info] ep$i: Set thumbnail has failed \n\n";
					}
					ivr_log_message($message);

					// End: Process featured image



					// Download images from remote
					if (!empty($content_img_linkAArr)) {
						foreach ($content_img_linkAArr as $key => $link) {
							$content_attachmt_id = ivr_attach_img_toPost($link, $insert_id, $i);

							if ($content_attachmt_id == 0) {
								$message = date('H:i:s'). "HKT [error] ep$i: Upload image failed -- $link \n";
								ivr_log_message($message);	
							} // end if


						} // end foreach

					} else {
						$message = date('H:i:s'). "HKT [info] ep$i: No content images. \n\n";
						ivr_log_message($message);

					}  // if-else !empty

					// End: Download images from remote


				} else {
					$message = date('H:i:s'). "HKT [error] ep$i: WP command failure -- wp_insert_post().\n";
					ivr_log_message($message);

				} // if-else !is_wp_error()
						
			} else {
				$message = date('H:i:s'). "HKT [error] ep$i: Failed to read json file.\n";
				ivr_log_message($message);
	
			}  //if-else read json

		}  //if-else check existence
		

	} // for loop

} else {

	?>
	<h5><br><br>Don't directly access this page.</h5>
	<div><a href="/php-import-script/"><p>Return</p></a></div>

<?php
}  // if Submit is set
?>

</div>  <!-- cu-container -->


<?php


/**
 * Find upload dir for image link from old site
 * 
 */


/**
 * Purpose: Attach remote image to post.  
 * 
 * If image already exists, attach the image to the post (Podcast) and return the attachment ID.
 * 
 * Otherwise, download the image from remote, upload to the corresponding
 * media directory, attach the image to the post (Podcast) and 
 * return the attachment ID.
 */
function ivr_attach_img_toPost($old_img_url, $post_id, $Num) {

	$image_infoArr = pathinfo($old_img_url); //Extracting information into array.
	$image_fname = $image_infoArr['basename'];
	$dirName = $image_infoArr['dirname'];
	$subDir = explode('uploads', $dirName)[1];
	$new_upload_baseDir = wp_upload_dir()['basedir'];
	$new_upload_dir = $new_upload_baseDir. $subDir;

	$new_img_url = get_site_url(). "/wp-content/uploads$subDir/$image_fname";

	if (wp_mkdir_p($new_upload_dir)) {
		$file = $new_upload_dir. '/'. $image_fname;
	} else {
		$file = $new_upload_baseDir . '/'. $image_fname;
	}

	// Check image file type
	$wp_filetype = wp_check_filetype($image_fname, null);

	// Set attachment data
	$attachment = array(
		'post_mime_type' => $wp_filetype['type'],
		'post_title' => sanitize_file_name($image_fname),
		'post_content' => '',
		'post_status' => 'inherit',
	);

	if ( !file_exists($file) ) {
		// Download remote image and create the image file on the server
		$image_data = file_get_contents($old_img_url);
		file_put_contents($file, $image_data);

		// Create the attachment
		$attach_id = wp_insert_attachment($attachment, $file, $post_id);

		// Define attachment metadata
		$attach_data = wp_generate_attachment_metadata($attach_id, $file);

		// Assign metadata to attachment
		wp_update_attachment_metadata($attach_id, $attach_data);

	} else {
		$attach_id = ivr_get_image_id($new_img_url);
		$message = date('H:i:s'). "HKT [info] ep$Num: Using existing content image (Att. ID: $attach_id)   \n";
		ivr_log_message($message);
	}

	return $attach_id;

} // ivr_attach_img_toPost()



/**
 *  retrieves the attachment ID from the file URL 
 * 
 * @param string $image_url 
 * @return int Attachment ID on success, 0 on failure
 * 
 * Ref: https://wpscholar.com/blog/get-attachment-id-from-wp-image-url/
 *  
 */
function ivr_get_image_id($image_url) {
	$attachment_id = 0;
	$dir = wp_upload_dir();

	if ( false !== strpos( $image_url, $dir['baseurl'] . '/' ) ) { // Is URL in uploads directory?
		$file = basename( $image_url );
		$query_args = array(
			'post_type'   => 'attachment',
			'post_status' => 'inherit',
			'fields'      => 'ids',
			'meta_query'  => array(
				array(
					'value'   => $file,
					'compare' => 'LIKE',
					'key'     => '_wp_attachment_metadata',
				),
			)
		);

		$query = new WP_Query( $query_args );

		if ( $query->have_posts() ) {

			foreach ( $query->posts as $post_id ) {
				$meta = wp_get_attachment_metadata( $post_id );
				$original_file  = basename( $meta['file'] );
				$cropped_image_files = wp_list_pluck( $meta['sizes'], 'file' );

				if ( $original_file === $file || in_array( $file, $cropped_image_files ) ) {
					$attachment_id = $post_id;
					break;
				}  // end if
			} // end foreach 

		} // end if have posts

	}

	return $attachment_id;

}  // ivr_get_image_id()



/**
 * Writes message logs to daily log file and displays logs online.
 */
function ivr_log_message($msg) {
	echo "<p style='color:#1b742e;margin:4px 0;'>". $msg. "</p>";
	$fName = date('Y-m-d'). "_HKT-import.log";
	file_put_contents( get_stylesheet_directory(). "/podcast_import/logs/$fName", $msg, FILE_APPEND);

} // ivr_log_message()



?>

<style>
	.cu-container {
		width: 80%;
		max-width: 900px;
		margin: 30px auto;
	}
</style>

Cookie	Duration	Description
cookielawinfo-checkbox-analytics	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional	11 months	The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy	11 months	The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.

Purpose

Main Steps

Export/Import

On cur-cwood

On DEV

On CWtdo

To follow up:

Codes on ‘cur-cwood’

page-php-script.php

page-php-result.php

Codes on ‘DEV’ and ‘CWtdo’

page-php-import-script.php

page-php-import-result.php

You Might Also Like

Align buttons across columns

Clare Wood Project Progress

Add Popup Banner on Top of Page Header